简单的爬虫

Posted nmucomputer

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了简单的爬虫相关的知识,希望对你有一定的参考价值。

对某一视频共享网站电影名进行爬取

 1 # Author:Winter Liu
 2 import time
 3 import urllib.request
 4 import re
 5 
 6 start_time = time.time()
 7 html_start = https://yanghuanyu.com/dy
 8 result = []
 9 for i in range(2,31):
10     response = urllib.request.urlopen(html_start)
11     buff = response.read()
12     html = buff.decode(utf-8)
13     # with open(‘hpage.txt‘,‘w‘,encoding=‘UTF-8‘) as f:
14     #     f.write(html)
15     print(html_start)
16     data = re.findall(r[.+][dddd][.+][.+], html)
17     data = list(set(data))
18     print(data)
19     result.extend(data)
20     html_start = "https://yanghuanyu.com/dy/page/"+str(i)
21 print(len(result))
22 print(result)
23 
24 end_time = time.time()
25 print(end_time  - start_time)

 

以上是关于简单的爬虫的主要内容,如果未能解决你的问题,请参考以下文章