爬虫环境

Posted 2020-11-09 mr-chenshuai

linux环境为例

 

安装python3

 

请求库

一、requests库

pipe install requests

注意：出现报错：pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.

安装以下俩库:

yum install openssl 


yum install openssl-devel

重新编译python3，python3默认没有支持ssl

cd Python-3.6.2

./configure --with-ssl

make

make install

 

二、selenium

pip3 install selenium==3.4.3（不跟版本默认安装最新版）

 

三、aiohttp

pip3 install aiohttp

 

解析库

一、lxml

pip3 install lxml

 

二、Beautiful Soup

pip3 install beautifulsoup4

 

三、pyquery

pip3 install pyquery