爬虫添加代理
Posted tuijin
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了爬虫添加代理相关的知识，希望对你有一定的参考价值。
# -*- coding: utf-8 -*- __author__ = ‘t_lishu‘ from urllib import request def use_proxy_ip(): url = ‘https://www.baidu.com/‘ proxy = {‘http‘: "10.41.82.82:8080"} # 创建ProxyHandler proxy_support = request.ProxyHandler(proxy) # 创建Opener opener = request.build_opener(proxy_support) # 添加User Angent opener.addheaders = [(‘user-agent‘, ‘Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (Khtml, like Gecko) Chrome/70.0.3538.77 Safari/537.36‘)] # 安装OPener request.install_opener(opener) response = request.urlopen(url) html = response.read() print(html) if __name__ == ‘__main__‘: use_proxy_ip()
以上是关于爬虫添加代理的主要内容，如果未能解决你的问题，请参考以下文章