Requests库基本使用

Posted yunlongaimeng

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Requests库基本使用相关的知识，希望对你有一定的参考价值。

Requests基于urllib3比urllib更简单方便。

基本的GET请求

1
2
3
4
import requests
 
response = requests.get(‘http://httpbin.org/get‘)
print(response.text)
技术分享图片 结果

带参数的GET请求

方式1

1
2
3
4
import requests
 
response = requests.get(‘http://httpbin.org/get?name=0bug&age=25‘)
print(response.text)
技术分享图片 结果

方式二

1
2
3
4
5
6
7
8
import requests
 
data = {
    ‘name‘: ‘0bug‘,
    ‘age‘: 25
}
response = requests.get(‘http://httpbin.org/get‘, params=data)
print(response.text)
技术分享图片 结果

解析Json

1
2
3
4
5
6
import requests
 
response = requests.get(‘http://httpbin.org/get‘)
print(type(response.text))
print(response.json())
print(type(response.json()))
技术分享图片 结果

获取二进制数据

1
2
3
4
5
6
7
import requests
 
response = requests.get(‘https://github.com/favicon.ico‘)
print(type(response.text))
print(type(response.content))
print(response.text)
print(response.content)
技术分享图片 结果

写入图片

1
2
3
4
5
import requests
 
response = requests.get(‘https://github.com/favicon.ico‘)
with open(‘img.ico‘,‘wb‘) as f:
    f.write(response.content)

添加headers

1
2
3
4
5
6
7
8
import requests
 
headers = {
    ‘User-Agent‘: ‘Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (Khtml, like Gecko) Chrome/66.0.3359.117 Safari/537.36‘
 
}
response = requests.get(‘https://www.baidu.com/‘, headers=headers)
print(response.status_code)

基本的POST请求

1
2
3
4
5
import requests
 
data = {‘name‘:‘0bug‘}
response = requests.post(‘http://httpbin.org/post‘,data=data)
print(response.text)
技术分享图片 结果
1
2
3
4
5
6
7
8
9
import requests
 
data = {‘name‘: ‘0bug‘}
headers = {
    ‘User-Agent‘: ‘Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.117 Safari/537.36‘
 
}
response = requests.post(‘http://httpbin.org/post‘, data=data, headers=headers)
print(response.json())
技术分享图片 结果

response属性

1
2
3
4
5
6
7
8
import requests
 
response = requests.get(‘http://www.jianshu.com‘)
print(type(response.status_code), response.status_code)
print(type(response.headers), response.headers)
print(type(response.cookies), response.cookies)
print(type(response.url), response.url)
print(type(response.history), response.history)
技术分享图片 结果

文件上传

1
2
3
4
5
import requests
 
files = {‘file‘: open(‘img.ico‘, ‘rb‘)}
response = requests.post(‘http://httpbin.org/post‘, files=files)
print(response.text)
技术分享图片 结果

获取cookie

1
2
3
4
5
6
import requests
 
response = requests.get(‘https://www.baidu.com‘)
print(response.cookies)
for key,value in response.cookies.items():
    print(key+‘=‘+value)
技术分享图片 结果

会话维持

1
2
3
4
5
6
import requests
 
s = requests.Session()
s.get(‘http://httpbin.org/cookies/set/number/123456‘)
response = s.get(‘http://httpbin.org/cookies‘)
print(response.text)
技术分享图片 结果

证书验证

1.无证书报错

1
2
3
4
import requests
 
response = requests.get(‘https://www.12306.cn‘)
print(response.status_code)

2.设置不使用证书，会返回200，也会有警告信息

1
2
3
4
import requests
 
response = requests.get(‘https://www.12306.cn‘,verify=False)
print(response.status_code)

3.消除警告信息

1
2
3
4
5
import requests
import urllib3
urllib3.disable_warnings()
response = requests.get(‘https://www.12306.cn‘,verify=False)
print(response.status_code)

4.使用本地证书

1
2
3
4
import requests
 
response = requests.get(‘https://www.12306.cn‘,cert=(‘/path/server.crt‘,‘/path/key‘))
print(response.status_code)

代理设置

1
2
3
4
5
6
7
8
9
import requests
 
proxies = {
    ‘http‘: ‘http://127.0.0.1:9743‘,
    ‘https‘: ‘https://127.0.0.1:9743‘
}
 
response = requests.get(‘https://www.taobao.com‘, proxies=proxies)
print(response.status_code)

有用户名和密码的代理

1
2
3
4
5
6
7
8
import requests
 
proxies = {
    ‘http‘: ‘http://user:[email protected]:9743‘,
}
 
response = requests.get(‘https://www.taobao.com‘, proxies=proxies)
print(response.status_code)

使用socks代理，需要安装一个模块

1
pip install requests[socks]

再使用代理

1
2
3
4
5
6
7
8
9
import requests
 
proxies = {
    ‘http‘: ‘socks5://127.0.0.1:9742‘,
    ‘https‘: ‘socks5://127.0.0.1:9742‘,
}
 
response = requests.get(‘https://www.taobao.com‘, proxies=proxies)
print(response.status_code)

超时设置

1
2
3
4
import requests
 
response = requests.get(‘https://www.taobao.com‘,timeout=0.01)
print(response.status_code)

错误处理

1
2
3
4
5
6
7
8
import requests
from requests.exceptions import ReadTimeout
 
try:
    response = requests.get(‘https://www.taobao.com‘, timeout=0.01)
    print(response.status_code)
except ReadTimeout:
    print(‘time out‘)

认证设置

方式1

1
2
3
4
5
import requests
from requests.auth import HTTPBasicAuth
 
r = requests.get(‘http://127.0.0.1:8080‘, auth=HTTPBasicAuth(‘user‘, ‘123‘))
print(r.status_code)

方式2

1
2
3
4
import requests
 
r = requests.get(‘http://127.0.0.1:8080‘, auth=(‘user‘, ‘123‘))
print(r.status_code)

异常处理

1
2
3
4
5
6
7
8
9
10
11
12
import requests
from requests.exceptions import ReadTimeout, HTTPError, RequestException
 
try:
    response = requests.get(‘http://httpbin.org/get‘, timeout=0.01)
    print(response.status_code)
except ReadTimeout:
    print(‘Timeout‘)
except HTTPError:
    print(‘HTTP REEOR‘)
except RequestException:
    print(‘Error‘)

以上是关于Requests库基本使用的主要内容，如果未能解决你的问题，请参考以下文章

Python-爬虫-基本库（requests）使用

Python:requests库BeautifulSoup4库的基本使用（实现简单的网络爬虫）

爬虫库requests和BeautifulSoup的基本使用

Python入门必学2个重点及精髓-Requests库~正则基本使用（上）

Python中requests库

从0开始学爬虫12之使用requests库基本认证