Urllib库

Posted 2021-01-09 cangshuchirou

urllib是一个包,这个包收集了几个用于处理URLs的模块

urllib.request      用于打开和读取URLs
urllib.error        用于触发请求的异常
urllib.parse        用于分析URLs
urllib.robotparser  用于分析robots.txt格式的文件

 

 

 

 

 

 URLOPEN练习

import urllib.request

response = urllib.request.urlopen("http://www.baidu.com")
print(response.read().decode("utf-8"))

第一个爬虫

import urllib.request
import urllib.parse

data = bytes(urllib.parse.urlencode({"word":"hello"}),encoding="utf8")
response = urllib.request.urlopen("http://httpbin.org/post",data=data)
print(response.read())

POST请求

import urllib.request

response = urllib.request.urlopen("http://httpbin.org/get",timeout=1)
print(response.read())

简单超时

import socket
import urllib.request
import urllib.error

try:
    response = urllib.request.urlopen("http://httpbin.org/get",timeout=0.1)
except urllib.error.URLError as e:
    if isinstance(e.reason,socket.timeout):
        print("TIME OUT")

简单的异常

 

 响应练习

 

import urllib.request

response = urllib.request.urlopen("https://www.python.org")
print(type(response))

响应类型

import urllib.request

response = urllib.request.urlopen("https://www.python.org")
print(response.status)
print(response.getheaders())
print(response.getheader("Server"))

获取状态码响应头

import urllib.request

request = urllib.request.Request("https://www.python.org")
response = urllib.request.urlopen(request)
print(response.read().decode("utf-8"))

得到响应内容