Python 3.x HTTP Error 403: Forbidden

Posted 2020-10-03

The Fobidden error often raised when using request.open to open some urls.

such as:

url_1 = ‘https://movie.douban.com/subject/26363254/comments?status=P‘
url_2 = ‘https://www.glassdoor.com/Interview/Texas-Instruments-Interview-Questions-E651_P4.htm‘

request.urlopen(url_1) # ** no error raised

request.urlopen(url_2) # ** Fobidden error raised

Here is the reason:

When using urllib.request.urlopen to visit a URL, the server will only receive a simple request for this webpage without knowing the hidden infos about exploer,operating system,platform, which are abnormal. 

Some websited will vefify the UserAgent info to prevent the abnoraml visisting. 

So the solution : Add these infos to the UserAgent to acts as using exploer to visit

req = request.Request(url,headers={‘User-Agent‘: ‘Mozilla/5.0‘}) # ** this would fix, also you can add other infos to User-Agent