一个完整的大作业

Posted 2020-10-13 46陈远钊

1.选一个自己感兴趣的主题。网址是：http://www.cntour.cn/travels/list.aspx?key=%E6%B5%B7%E5%8D%97，搜海南的游记。

 

2.网络上爬取相关的数据。

import requests
from bs4 import BeautifulSoup
import re
url=\'http://www.cntour.cn/travels/list.aspx?key=%E6%B5%B7%E5%8D%97\'
res=requests.get(url)
res.encoding=\'utf-8\'
soup=BeautifulSoup(res.text,\'html.parser\')

for trips in soup.select(\'ul\'):
    if len(trips.select(\'.title\'))>0:
      title=trips.select(\'.title\')[0].text
      url=trips.select(\'a\')[0][\'href\']
      
      print(title,url)

 

3.进行文本分析，生成词云。

import jieba
from wordcloud import WordCloud
import matplotlib.pyplot as plt
import requests
from bs4 import BeautifulSoup
import re

youji=open("/Users/游记.txt","r",encoding=\'gb2312\').read()
words=jieba.lcut(youji)
counts={}
ls=[]
for word in words:
    ls.append(word)
    if len(word)==1:
        continue
    else:
        counts[word]=counts.get(word,0)+1
items=list(counts.items())
items.sort(key=lambda x:x[1],reverse=True)
for i in range(20):
    word,count=items[i]
    print("{0:<10}{1:>5}".format(word,count))
    
wordlist=jieba.cut(youji, cut_all=True)
wl_split="/".join(wordlist)
cy=WordCloud(font_path=r\'C:\\Users\\Administrator\\Desktop\\simheittf\\simhei.ttf\').generate(wl_split)
plt.imshow(cy)
plt.axis("off")
plt.show()

 

从词云可以看出，人们去海南旅游大都去的是三亚，海鲜、美景、享受等是那里的关键词。