爬虫添加代理

Posted tuijin

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了爬虫添加代理相关的知识，希望对你有一定的参考价值。

# -*- coding: utf-8 -*-
__author__ = ‘t_lishu‘

from urllib import request

def use_proxy_ip():
    url = ‘https://www.baidu.com/‘
    proxy = {‘http‘: "10.41.82.82:8080"}
    # 创建ProxyHandler
    proxy_support = request.ProxyHandler(proxy)
    # 创建Opener
    opener = request.build_opener(proxy_support)
    # 添加User Angent
    opener.addheaders = [(‘user-agent‘, ‘Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (Khtml, like Gecko) Chrome/70.0.3538.77 Safari/537.36‘)]
    # 安装OPener
    request.install_opener(opener)
    response = request.urlopen(url)
    html = response.read()
    print(html)
if __name__ == ‘__main__‘:
    use_proxy_ip()

 

以上是关于爬虫添加代理的主要内容，如果未能解决你的问题，请参考以下文章