文件方式实现完整的英文词频统计实例

Posted 2020-10-08 xy223

可以下载一长篇的英文小说，进行词频的分析。

1.读入待分析的字符串

2.分解提取单词 

3.计数字典

4.排除语法型词汇

5.排序

6.输出TOP(20)

7.对输出结果的简要说明。

ordinary=open(\'ordinary.txt\',\'w\')
ordinary.write(\'\'\'This isjust an ordinary day,
Wipe the insecurities away. 
I can see that the darkness will erode, 
Looking out the corner of my eye,.
I can see that the sunshine will explode,
Far across the desert in the sky. 
Beautiful girl won\'t you be my inspiration, 
Beautiful girl don\'t you throw your love around.
What in the world, what in the world could ever come between us,
beautiful girl, beautiful girl. 
I\'ll never let you down, won\'t let you down, 
This is the beginning of your day. 
Life is more intricate than it seems, 
Always be yourself along the way. 
Living through the spirit of your dreams.\'\'\')
ordinary.close()
comment=open(\'ordinary.txt\',\'r\')
ordinary=comment.read()
comment.close()
 
ordinary=ordinary.lower()
for i in ",.?!()":
    ordinary=ordinary.replace(i,\' \')
ordinary=ordinary.replace(\'\\n\',\' \')
words=ordinary.split(\' \')
s=set(words)
 
delete={"is","an","the","in"}
for i in delete:
    s.remove(i)
     
dic={}
lis=[]
for i in s:
    if(i==" "):
        continue
    if(i==""):
        continue
    dic[i]=words.count(i)
    lis.append(words.count(i))
 
lis=list (dic.items())
lis.sort(key=lambda x:x[1],reverse=True)
for i in range(20):
    print(lis[i])