综合练习：英文词频统计

Posted 2020-11-08 xichenmm

• 词频统计预处理
• 下载一首英文的歌词或文章
• 将所有,.？！’:等分隔符全部替换为空格
• 将所有大写转换为小写
• 生成单词列表
• 生成词频统计
• 排序
• 排除语法型词汇，代词、冠词、连词
• 输出词频最大TOP10

• # -*- coding: UTF-8 -*-
  # -*- author: yjw -*-

  Music="""Oh if there‘s one thing to be taught
  It‘s dreams are made to be caught
  And friends can never be bought
  Doesn‘t matter how long it‘s been
  I know you‘ll always jump in
  ‘Cause we don‘t know how to quit
  Let‘s start a riot tonight
  A pack of lions tonight
  In this world, he who stops, won‘t get anything he wants
  Play like the top one percent
  Til nothing‘s left to be spent
  Take it all, ours to take, celebrate because
  We are the champions
  Setting it off again
  Oh we on fire
  We on fire
  Running our own campaign
  Doing the whole shebang
  Oh we on fire
  We on fire
  Get up, stand up, throw your hands up
  Welcome to the other land of
  Dreamers brothers sisters others
  Yeah we on fire like that
  Oh the bond is deeper than skin
  The kind of club that we‘re in
  The kind of love that we give
  Oh ever since the dawn of mankind
  We all belong to a tribe
  It‘s good to know this one‘s mine
  Let‘s start a riot tonight
  A pack of lions tonight
  In this world, he who stops, won‘t get anything he wants
  Play like the top one percent
  Til nothing‘s left to be spent
  We don‘t care, we won‘t stop, call your mothers, call the cops
  We are the champions
  Setting it off again
  Oh we on fire
  We on fire
  Running our own campaign
  Doing the whole shebang
  Oh we on fire
  We on fire
  Get up, stand up, throw your hands up
  Welcome to the other land of
  Dreamers brothers sisters others
  Yeah we on fire like that
  Ohh (fire, fire)
  Ooooh (fire, fire)
  And nothing‘s gonna be the same (fire fire, fire fire, fire fire)
  Oh! We‘re the champions
  We are the champions
  Setting it off again
  Oh we on fire
  We on fire
  Running our own campaign
  Doing the whole shebang
  Oh we on fire
  We on fire
  Dit dit heart and soul
  Hey and nothing‘s going to be the same
  Hey the life that you made will not be today"""

  Sign=[‘,‘,‘.‘,‘?‘,‘!‘,‘(‘,‘)‘,"‘"]
  article=[‘a‘,‘in‘,‘of‘,‘the‘,‘to‘,‘at‘,‘it‘,‘on‘,‘and‘,‘so‘,‘his‘,‘that‘,‘not‘,‘was‘,‘my‘,‘were‘,‘we‘,‘he‘,‘an‘,‘as‘,‘is‘,‘for‘,‘mr‘,‘us‘,‘oh‘]

  for x in Sign:
  Music2=Music.replace(x,‘‘)

  Music2=Music2.lower()
  Music2=Music2.split()

  art={}
  for y in Music2:
  frequency=Music2.count(y)
  art[y]=frequency

  for z in article:
  if z in art.keys():
  del art[z]

  art=sorted(art.items(),key=lambda item:item[1],reverse=True)
  for i in range(10):
  print(art[i])