设置随机请求头和使用代理
Posted zxfei
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了设置随机请求头和使用代理相关的知识，希望对你有一定的参考价值。
# -*- coding: utf-8 -*-
"""
所有请求头的USER_AGENTS网址
http://www.useragentstring.com/pages/useragentstring.php?name=All
"""
import json
import random
import requests
USER_AGENTS = [
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (Khtml, like Gecko) Chrome/70.0.3538.77 Safari/537.36',
'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.88 Safari/537.36',
'Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/44.0.2403.155 Safari/537.36',
'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36',
'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.1 Safari/537.36 Chrome 41.0.2227.0',
'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2227.0 Safari/537.36',
'Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2226.0 Safari/537.36',
'Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2225.0 Safari/537.36',
]
#从列表中随机选择一个
User_Agent = random.choice(USER_AGENTS)
headers = {
'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language': 'en',
'User-Agent': User_Agent
}
def testUserAgent():
resp = requests.get("http://httpbin.org/user-agent", headers=headers)
user_agent = json.loads(resp.text)['user-agent']#将返回的字符串loads为字典
print(user_agent)
def testProxy():
proxies = {"https": "223.199.25.183:9999"}
resp = requests.get("http://httpbin.org/ip", proxies=proxies)
origin = json.loads(resp.text)['origin']
print(origin)
# https://www.xicidaili.com/nn/ 西刺代理
if __name__ == "__main__":
testUserAgent()
testProxy()
以上是关于设置随机请求头和使用代理的主要内容，如果未能解决你的问题，请参考以下文章