python爬虫，使用requests设置代理

Posted 2021-06-15 布都御魂

免费代理的网站: http://www.xicidaili.com/nn/

代码部分：

import requests

proxy=\'124.243.226.18:8888\'

#如果代理需要验证，只需要在前面加上用户名密码，如下所示

# proxy=\'username:password@124.243.226.18:8888\'
proxies={
    \'http\':\'http://\'+proxy,
    \'https\':\'https://\'+proxy,
}
try:
    response=requests.get(\'http://httpbin.org/get\',proxies=proxies)
    print(response.text)
except requests.exceptions.ConnectionError as e:
    print("Error",e.args)
 

输出：

{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Accept-Encoding": "gzip, deflate",
    "Connection": "close",
    "Host": "httpbin.org",
    "User-Agent": "python-requests/2.20.0"
  },
  "origin": "124.243.226.18",
  "url": "http://httpbin.org/get"
}

 

 

基于 selenium的代理设置：

from selenium import webdriver

proxy=\'124.243.226.18:8888\'

option=webdriver.ChromeOptions()

option.add_argument(\'--proxy-server=http://\'+proxy)

driver = webdriver.Chrome(options=option)

driver.get(\'http://httpbin.org/get\')