NLTK
这是一个处理文本的python库，我们知道文字性的知识可是拥有非常庞大的数据量，故而这属于大数据系列。
本文只是浅尝辄止，目前本人并未涉及这块知识，只是偶尔好奇，才写本文。
从NLTK中的book模块中，载入所有条目
- book 模块包含所有数据
from nltk.book import *
*** Introductory Examples for the NLTK Book ***
Loading text1, ..., text9 and sent1, ..., sent9
Type the name of the text or sentence to view it.
Type: \'texts()\' or \'sents()\' to list the materials.
text1: Moby Dick by Herman Melville 1851
text2: Sense and Sensibility by Jane Austen 1811
text3: The Book of Genesis
text4: Inaugural Address Corpus
text5: Chat Corpus
text6: Monty Python and the Holy Grail
text7: Wall Street Journal
text8: Personals Corpus
text9: The Man Who Was Thursday by G . K . Chesterton 1908
text1
<Text: Moby Dick by Herman Melville 1851>
text2
<Text: Sense and Sensibility by Jane Austen 1811>
搜索文本或主题
- concordance允许在课文中查找单词，并打印出来
- similar 用来识别文章中和搜索词相似的词语，可以用在搜索引擎中的相关度识别功能中。
- common_contexts 用来识别2个关键词相似的词语。
- dispersion_plot 绘制单词的离散图
text1.concordance(\'monstrous\') # 在text1中查阅词汇\'monstrous\'
# concordance
# 英 [kən\'kɔːd(ə)ns] 美 [kən\'kɔrdns]
# n. 调和，一致；用语索引；著作或作家全集的重要用字索引
Displaying 11 of 11 matches:
ong the former , one was of a most monstrous size . ... This came towards us ,
ON OF THE PSALMS . " Touching that monstrous bulk of the whale or ork we have r
ll over with a heathenish array of monstrous clubs and spears . Some were thick
d as you gazed , and wondered what monstrous cannibal and savage could ever hav
that has survived the flood ; most monstrous and most mountainous ! That Himmal
they might scout at Moby Dick as a monstrous fable , or still worse and more de
th of Radney .\'" CHAPTER 55 Of the Monstrous Pictures of Whales . I shall ere l
ing Scenes . In connexion with the monstrous pictures of whales , I am strongly
ere to enter upon those still more monstrous stories of them which are to be fo
ght have been rummaged out of this monstrous cabinet there is no telling . But
of Whale - Bones ; for Whales of a monstrous size are oftentimes cast up dead u
text2.concordance(\'affection\')
Displaying 25 of 79 matches:
, however , and , as a mark of his affection for the three girls , he left them
t . It was very well known that no affection was ever supposed to exist between
deration of politeness or maternal affection on the side of the former , the tw
d the suspicion -- the hope of his affection for me may warrant , without impru
hich forbade the indulgence of his affection . She knew that his mother neither
rd she gave one with still greater affection . Though her late conversation wit
can never hope to feel or inspire affection again , and if her home be uncomfo
m of the sense , elegance , mutual affection , and domestic comfort of the fami
, and which recommended him to her affection beyond every thing else . His soci
ween the parties might forward the affection of Mr . Willoughby , an equally st
the most pointed assurance of her affection . Elinor could not be surprised at
he natural consequence of a strong affection in a young and ardent mind . This
opinion . But by an appeal to her affection for her mother , by representing t
every alteration of a place which affection had established as perfect with hi
e will always have one claim of my affection , which no other can possibly shar
f the evening declared at once his affection and happiness . " Shall we see you
ause he took leave of us with less affection than his usual behaviour has shewn
ness ." " I want no proof of their affection ," said Elinor ; " but of their en
onths , without telling her of his affection ;-- that they should part without
ould be the natural result of your affection for her . She used to be all unres
distinguished Elinor by no mark of affection . Marianne saw and listened with i
th no inclination for expense , no affection for strangers , no profession , an
till distinguished her by the same affection which once she had felt no doubt o
al of her confidence in Edward \' s affection , to the remembrance of every mark
was made ? Had he never owned his affection to yourself ?" " Oh , no ; but if
text1.similar(\'monstrous\')
true contemptible christian abundant few part mean careful puzzled
mystifying passing curious loving wise doleful gamesome singular
delightfully perilous fearless
text2.similar(\'monstrous\')
very so exceedingly heartily a as good great extremely remarkably
sweet vast amazingly
text2.common_contexts([\'monstrous\',\'very\'])
a_pretty am_glad a_lucky is_pretty be_glad
# 从文本中检查一个单词的位置，从该单词出现开始出现了多少次。
# Each stripe represents an instance of a word,
# and each row represents the entire text.
text4.dispersion_plot([\'citizens\',\'democracy\',\'freedon\',\'duties\',\'America\',\'liberty\'])
# dispersion
# 英 [dɪ\'spɜːʃ(ə)n] 美 [dɪ\'spɝʒn]
# n. 散布；[统计][数] 离差；驱散
print(text3.generate(\'monstrous\'))
None
统计词汇
len(text3)
44764
sorted(set(text3))
[\'!\',
"\'",
\'(\',
\')\',
\',\',
\',)\',
\'.\',
\'.)\',
\':\',
\';\',
\';)\',
\'?\',
\'?)\',
\'A\',
\'Abel\',
\'Abelmizraim\',
\'Abidah\',
\'Abide\',
\'Abimael\',
\'Abimelech\',
\'Abr\',
\'Abrah\',
\'Abraham\',
\'Abram\',
\'Accad\',
\'Achbor\',
\'Adah\',
\'Adam\',
\'Adbeel\',
\'Admah\',
\'Adullamite\',
\'After\',
\'Aholibamah\',
\'Ahuzzath\',
\'Ajah\',
\'Akan\',
\'All\',
\'Allonbachuth\',
\'Almighty\',
\'Almodad\',
\'Also\',
\'Alvah\',
\'Alvan\',
\'Am\',
\'Amal\',
\'Amalek\',
\'Amalekites\',
\'Ammon\',
\'Amorite\',
\'Amorites\',
\'Amraphel\',
\'An\',
\'Anah\',
\'Anamim\',
\'And\',
\'Aner\',
\'Angel\',
\'Appoint\',
\'Aram\',
\'Aran\',
\'Ararat\',
\'Arbah\',
\'Ard\',
\'Are\',
\'Areli\',
\'Arioch\',
\'Arise\',
\'Arkite\',
\'Arodi\',
\'Arphaxad\',
\'Art\',
\'Arvadite\',
\'As\',
\'Asenath\',
\'Ashbel\',
\'Asher\',
\'Ashkenaz\',
\'Ashteroth\',
\'Ask\',
\'Asshur\',
\'Asshurim\',
\'Assyr\',
\'Assyria\',
\'At\',
\'Atad\',
\'Avith\',
\'Baalhanan\',
\'Babel\',
\'Bashemath\',
\'Be\',
\'Because\',
\'Becher\',
\'Bedad\',
\'Beeri\',
\'Beerlahairoi\',
\'Beersheba\',
\'Behold\',
\'Bela\',
\'Belah\',
\'Benam\',
\'Benjamin\',
\'Beno\',
\'Beor\',
\'Bera\',
\'Bered\',
\'Beriah\',
\'Bethel\',
\'Bethlehem\',
\'Bethuel\',
\'Beware\',
\'Bilhah\',
\'Bilhan\',
\'Binding\',
\'Birsha\',
\'Bless\',
\'Blessed\',
\'Both\',
\'Bow\',
\'Bozrah\',
\'Bring\',
\'But\',
\'Buz\',
\'By\',
\'Cain\',
\'Cainan\',
\'Calah\',
\'Calneh\',
\'Can\',
\'Cana\',
\'Canaan\',
\'Canaanite\',
\'Canaanites\',
\'Canaanitish\',
\'Caphtorim\',
\'Carmi\',
\'Casluhim\',
\'Cast\',
\'Cause\',
\'Chaldees\',
\'Chedorlaomer\',
\'Cheran\',
\'Cherubims\',
\'Chesed\',
\'Chezib\',
\'Come\',
\'Cursed\',
\'Cush\',
\'Damascus\',
\'Dan\',
\'Day\',
\'Deborah\',
\'Dedan\',
\'Deliver\',
\'Diklah\',
\'Din\',
\'Dinah\',
\'Dinhabah\',
\'Discern\',
\'Dishan\',
\'Dishon\',
\'Do\',
\'Dodanim\',
\'Dothan\',
\'Drink\',
\'Duke\',
\'Dumah\',
\'Earth\',
\'Ebal\',
\'Eber\',
\'Edar\',
\'Eden\',
\'Edom\',
\'Edomites\',
\'Egy\',
\'Egypt\',
\'Egyptia\',
\'Egyptian\',
\'Egyptians\',
\'Ehi\',
\'Elah\',
\'Elam\',
\'Elbethel\',
\'Eldaah\',
\'EleloheIsrael\',
\'Eliezer\',
\'Eliphaz\',
\'Elishah\',
\'Ellasar\',
\'Elon\',
\'Elparan\',
\'Emins\',
\'En\',
\'Enmishpat\',
\'Eno\',
\'Enoch\',
\'Enos\',
\'Ephah\',
\'Epher\',
\'Ephra\',
\'Ephraim\',
\'Ephrath\',
\'Ephron\',
\'Er\',
\'Erech\',
\'Eri\',
\'Es\',
\'Esau\',
\'Escape\',
\'Esek\',
\'Eshban\',
\'Eshcol\',
\'Ethiopia\',
\'Euphrat\',
\'Euphrates\',
\'Eve\',
\'Even\',
\'Every\',
\'Except\',
\'Ezbon\',
\'Ezer\',
\'Fear\',
\'Feed\',
\'Fifteen\',
\'Fill\',
\'For\',
\'Forasmuch\',
\'Forgive\',
\'From\',
\'Fulfil\',
\'G\',
\'Gad\',
\'Gaham\',
\'Galeed\',
\'Gatam\',
\'Gather\',
\'Gaza\',
\'Gentiles\',
\'Gera\',
\'Gerar\',
\'Gershon\',
\'Get\',
\'Gether\',
\'Gihon\',
\'Gilead\',
\'Girgashites\',
\'Girgasite\',
\'Give\',
\'Go\',
\'God\',
\'Gomer\',
\'Gomorrah\',
\'Goshen\',
\'Guni\',
\'Hadad\',
\'Hadar\',
\'Hadoram\',
\'Hagar\',
\'Haggi\',
\'Hai\',
\'Ham\',
\'Hamathite\',
\'Hamor\',
\'Hamul\',
\'Hanoch\',
\'Happy\',
\'Haran\',
\'Hast\',
\'Haste\',
\'Have\',
\'Havilah\',
\'Hazarmaveth\',
\'Hazezontamar\',
\'Hazo\',
\'He\',
\'Hear\',
\'Heaven\',
\'Heber\',
\'Hebrew\',
\'Hebrews\',
\'Hebron\',
\'Hemam\',
\'Hemdan\',
\'Here\',
\'Hereby\',
\'Heth\',
\'Hezron\',
\'Hiddekel\',
\'Hinder\',
\'Hirah\',
\'His\',
\'Hitti\',
\'Hittite\',
\'Hittites\',
\'Hivite\',
\'Hobah\',
\'Hori\',
\'Horite\',
\'Horites\',
\'How\',
\'Hul\',
\'Huppim\',
\'Husham\',
\'Hushim\',
\'Huz\',
\'I\',
\'If\',
\'In\',
\'Irad\',
\'Iram\',
\'Is\',
\'Isa\',
\'Isaac\',
\'Iscah\',
\'Ishbak\',
\'Ishmael\',
\'Ishmeelites\',
\'Ishuah\',
\'Isra\',
\'Israel\',
\'Issachar\',
\'Isui\',
\'It\',
\'Ithran\',
\'Jaalam\',
\'Jabal\',
\'Jabbok\',
\'Jac\',
\'Jachin\',
\'Jacob\',
\'Jahleel\',
\'Jahzeel\',
\'Jamin\',
\'Japhe\',
\'Japheth\',
\'Jared\',
\'Javan\',
\'Jebusite\',
\'Jebusites\',
\'Jegarsahadutha\',
\'Jehovahjireh\',
\'Jemuel\',
\'Jerah\',
\'Jetheth\',
\'Jetur\',
\'Jeush\',
\'Jezer\',
\'Jidlaph\',
\'Jimnah\',
\'Job\',
\'Jobab\',
\'Jokshan\',
\'Joktan\',
\'Jordan\',
\'Joseph\',
\'Jubal\',
\'Judah\',
\'Judge\',
\'Judith\',
\'Kadesh\',
\'Kadmonites\',
\'Karnaim\',
\'Kedar\',
\'Kedemah\',
\'Kemuel\',
\'Kenaz\',
\'Kenites\',
\'Kenizzites\',
\'Keturah\',
\'Kiriathaim\',
\'Kirjatharba\',
\'Kittim\',
\'Know\',
\'Kohath\',
\'Kor\',
\'Korah\',
\'LO\',
\'LORD\',
\'Laban\',
\'Lahairoi\',
\'Lamech\',
\'Lasha\',
\'Lay\',
\'Leah\',
\'Lehabim\',
\'Lest\',
\'Let\',
\'Letushim\',
\'Leummim\',
\'Levi\',
\'Lie\',
\'Lift\',
\'Lo\',
\'Look\',
\'Lot\',
\'Lotan\',
\'Lud\',
\'Ludim\',
\'Luz\',
\'Maachah\',
\'Machir\',
\'Machpelah\',
\'Madai\',
\'Magdiel\',
\'Magog\',
\'Mahalaleel\',
\'Mahalath\',
\'Mahanaim\',
\'Make\',
\'Malchiel\',
\'Male\',
\'Mam\',
\'Mamre\',
\'Man\',
\'Manahath\',
\'Manass\',
\'Manasseh\',
\'Mash\',
\'Masrekah\',
\'Massa\',
\'Matred\',
\'Me\',
\'Medan\',
\'Mehetabel\',
\'Mehujael\',
\'Melchizedek\',
\'Merari\',
\'Mesha\',
\'Meshech\',
\'Mesopotamia\',
\'Methusa\',
\'Methusael\',
\'Methuselah\',
\'Mezahab\',
\'Mibsam\',
\'Mibzar\',
\'Midian\',
\'Midianites\',
\'Milcah\',
\'Mishma\',
\'Mizpah\',
\'Mizraim\',
\'Mizz\',
\'Moab\',
\'Moabites\',
\'Moreh\',
\'Moreover\',
\'Moriah\',
\'Muppim\',
\'My\',
\'Naamah\',
\'Naaman\',
\'Nahath\',
\'Nahor\',
\'Naphish\',
\'Naphtali\',
\'Naphtuhim\',
\'Nay\',
\'Nebajoth\',
\'Neither\',
\'Night\',
\'Nimrod\',
\'Nineveh\',
\'Noah\',
\'Nod\',
\'Not\',
\'Now\',
\'O\',
\'Obal\',
\'Of\',
\'Oh\',
\'Ohad\',
\'Omar\',
\'On\',
\'Onam\',
\'Onan\',
\'Only\',
\'Ophir\',
\'Our\',
\'Out\',
\'Padan\',
\'Padanaram\',
\'Paran\',
\'Pass\',
\'Pathrusim\',
\'Pau\',
\'Peace\',
\'Peleg\',
\'Peniel\',
\'Penuel\',
\'Peradventure\',
\'Perizzit\',
\'Perizzite\',
\'Perizzites\',
\'Phallu\',
\'Phara\',
\'Pharaoh\',
\'Pharez\',
\'Phichol\',
\'Philistim\',
\'Philistines\',
\'Phut\',
\'Phuvah\',
\'Pildash\',
\'Pinon\',
\'Pison\',
\'Potiphar\',
\'Potipherah\',
\'Put\',
\'Raamah\',
\'Rachel\',
\'Rameses\',
\'Rebek\',
\'Rebekah\',
\'Rehoboth\',
\'Remain\',
\'Rephaims\',
\'Resen\',
\'Return\',
\'Reu\',
\'Reub\',
\'Reuben\',
\'Reuel\',
\'Reumah\',
\'Riphath\',
\'Rosh\',
\'Sabtah\',
\'Sabtech\',
\'Said\',
\'Salah\',
\'Salem\',
\'Samlah\',
\'Sarah\',
\'Sarai\',
\'Saul\',
\'Save\',
\'Say\',
\'Se\',
\'Seba\',
\'See\',
\'Seeing\',
\'Seir\',
\'Sell\',
\'Send\',
\'Sephar\',
\'Serah\',
\'Sered\',
\'Serug\',
\'Set\',
\'Seth\',
\'Shalem\',
\'Shall\',
\'Shalt\',
\'Shammah\',
\'Shaul\',
\'Shaveh\',
\'She\',
\'Sheba\',
\'Shebah\',
\'Shechem\',
\'Shed\',
\'Shel\',
\'Shelah\',
\'Sheleph\',
\'Shem\',
\'Shemeber\',
\'Shepho\',
\'Shillem\',
\'Shiloh\',
\'Shimron\',
\'Shinab\',
\'Shinar\',
\'Shobal\',
\'Should\',
\'Shuah\',
\'Shuni\',
\'Shur\',
\'Sichem\',
\'Siddim\',
\'Sidon\',
\'Simeon\',
\'Sinite\',
\'Sitnah\',
\'Slay\',
\'So\',
\'Sod\',
\'Sodom\',
\'Sojourn\',
\'Some\',
\'Spake\',
\'Speak\',
\'Spirit\',
\'Stand\',
\'Succoth\',
\'Surely\',
\'Swear\',
\'Syrian\',
\'Take\',
\'Tamar\',
\'Tarshish\',
\'Tebah\',
\'Tell\',
\'Tema\',
\'Teman\',
\'Temani\',
\'Terah\',
\'Thahash\',
\'That\',
\'The\',
\'Then\',
\'There\',
\'Therefore\',
\'These\',
\'They\',
\'Thirty\',
\'This\',
\'Thorns\',
\'Thou\',
\'Thus\',
\'Thy\',
\'Tidal\',
\'Timna\',
\'Timnah\',
\'Timnath\',
\'Tiras\',
\'To\',
\'Togarmah\',
\'Tola\',
\'Tubal\',
\'Tubalcain\',
\'Twelve\',
\'Two\',
\'Unstable\',
\'Until\',
\'Unto\',
\'Up\',
\'Upon\',
\'Ur\',
\'Uz\',
\'Uzal\',
\'We\',
\'What\',
\'When\',
\'Whence\',
\'Where\',
\'Whereas\',
\'Wherefore\',
\'Which\',
\'While\',
\'Who\',
\'Whose\',
\'Whoso\',
\'Why\',
\'Wilt\',
\'With\',
\'Woman\',
\'Ye\',
\'Yea\',
\'Yet\',
\'Zaavan\',
\'Zaphnathpaaneah\',
\'Zar\',
\'Zarah\',
\'Zeboiim\',
\'Zeboim\',
\'Zebul\',
\'Zebulun\',
\'Zemarite\',
\'Zepho\',
\'Zerah\',
\'Zibeon\',
\'Zidon\',
\'Zillah\',
\'Zilpah\',
\'Zimran\',
\'Ziphion\',
\'Zo\',
\'Zoar\',
\'Zohar\',
\'Zuzims\',
\'a\',
\'abated\',
\'abide\',
\'able\',
\'abode\',
\'abomination\',
\'about\',
\'above\',
\'abroad\',
\'absent\',
\'abundantly\',
\'accept\',
\'accepted\',
\'according\',
\'acknowledged\',
\'activity\',
\'add\',
\'adder\',
\'afar\',
\'afflict\',
\'affliction\',
\'afraid\',
\'after\',
\'afterward\',
\'afterwards\',
\'aga\',
\'again\',
\'against\',
\'age\',
\'aileth\',
\'air\',
\'al\',
\'alive\',
\'all\',
\'almon\',
\'alo\',
\'alone\',
\'aloud\',
\'also\',
\'altar\',
\'altogether\',
\'always\',
\'am\',
\'among\',
\'amongst\',
\'an\',
\'and\',
\'angel\',
\'angels\',
\'anger\',
\'angry\',
\'anguish\',
\'anointedst\',
\'anoth\',
\'another\',
\'answer\',
\'answered\',
\'any\',
\'anything\',
\'appe\',
\'appear\',
\'appeared\',
\'appease\',
\'appoint\',
\'appointed\',
\'aprons\',
\'archer\',
\'archers\',
\'are\',
\'arise\',
\'ark\',
\'armed\',
\'arms\',
\'army\',
\'arose\',
\'arrayed\',
\'art\',
\'artificer\',
\'as\',
\'ascending\',
\'ash\',
\'ashamed\',
\'ask\',
\'asked\',
\'asketh\',
\'ass\',
\'assembly\',
\'asses\',
\'assigned\',
\'asswaged\',
\'at\',
\'attained\',
\'audience\',
\'avenged\',
\'aw\',
\'awaked\',
\'away\',
\'awoke\',
\'back\',
\'backward\',
\'bad\',
\'bade\',
\'badest\',
\'badne\',
\'bak\',
\'bake\',
\'bakemeats\',
\'baker\',
\'bakers\',
\'balm\',
\'bands\',
\'bank\',
\'bare\',
\'barr\',
\'barren\',
\'basket\',
\'baskets\',
\'battle\',
\'bdellium\',
\'be\',
\'bear\',
\'beari\',
\'bearing\',
\'beast\',
\'beasts\',
\'beautiful\',
\'became\',
\'because\',
\'become\',
\'bed\',
\'been\',
\'befall\',
\'befell\',
\'before\',
\'began\',
\'begat\',
\'beget\',
\'begettest\',
\'begin\',
\'beginning\',
\'begotten\',
\'beguiled\',
\'beheld\',
\'behind\',
\'behold\',
\'being\',
\'believed\',
\'belly\',
\'belong\',
\'beneath\',
\'bereaved\',
\'beside\',
\'besides\',
\'besought\',
\'best\',
\'betimes\',
\'better\',
\'between\',
\'betwixt\',
\'beyond\',
\'binding\',
\'bird\',
\'birds\',
\'birthday\',
\'birthright\',
\'biteth\',
\'bitter\',
\'blame\',
\'blameless\',
\'blasted\',
\'bless\',
\'blessed\',
\'blesseth\',
\'blessi\',
\'blessing\',
\'blessings\',
\'blindness\',
\'blood\',
\'blossoms\',
\'bodies\',
\'boldly\',
\'bondman\',
\'bondmen\',
\'bondwoman\',
\'bone\',
\'bones\',
\'book\',
\'booths\',
\'border\',
\'borders\',
\'born\',
\'bosom\',
\'both\',
\'bottle\',
\'bou\',
\'boug\',
\'bough\',
\'bought\',
\'bound\',
\'bow\',
\'bowed\',
\'bowels\',
\'bowing\',
\'boys\',
\'bracelets\',
\'branches\',
\'brass\',
\'bre\',
\'breach\',
\'bread\',
\'breadth\',
\'break\',
\'breaketh\',
\'breaking\',
\'breasts\',
\'breath\',
\'breathed\',
\'breed\',
\'brethren\',
\'brick\',
\'brimstone\',
\'bring\',
\'brink\',
\'broken\',
\'brook\',
\'broth\',
\'brother\',
\'brought\',
\'brown\',
\'bruise\',
\'budded\',
\'build\',
\'builded\',
\'built\',
\'bulls\',
\'bundle\',
\'bundles\',
\'burdens\',
\'buried\',
\'burn\',
\'burning\',
\'burnt\',
\'bury\',
\'buryingplace\',
\'business\',
\'but\',
\'butler\',
\'butlers\',
\'butlership\',
\'butter\',
\'buy\',
\'by\',
\'cakes\',
\'calf\',
\'call\',
\'called\',
\'came\',
\'camel\',
\'camels\',
\'camest\',
\'can\',
\'cannot\',
\'canst\',
\'captain\',
\'captive\',
\'captives\',
\'carcases\',
\'carried\',
\'carry\',
\'cast\',
\'castles\',
\'catt\',
\'cattle\',
\'caught\',
\'cause\',
\'caused\',
\'cave\',
\'cease\',
\'ceased\',
\'certain\',
\'certainly\',
\'chain\',
\'chamber\',
\'change\',
\'changed\',
\'changes\',
\'charge\',
\'charged\',
\'chariot\',
\'chariots\',
\'chesnut\',
\'chi\',
\'chief\',
\'child\',
\'childless\',
\'childr\',
\'children\',
\'chode\',
\'choice\',
\'chose\',
\'circumcis\',
\'circumcise\',
\'circumcised\',
\'citi\',
\'cities\',
\'city\',
\'clave\',
\'clean\',
\'clear\',
\'cleave\',
\'clo\',
\'closed\',
\'clothed\',
\'clothes\',
\'cloud\',
\'clusters\',
\'co\',
\'coat\',
\'coats\',
\'coffin\',
\'cold\',
...]
len(set(text3))
2789
len(text3)/len(set(text3))
16.050197203298673
text3.count(\'smote\')
5
100*text4.count(\'a\')/len(text4)
1.4643016433938312
def lexical_diversity(text):
# lexical英[\'leksɪk(ə)l] 美 [\'lɛksɪkl]
# adj.词汇的；[语] 词典的；词典编纂的
# diversity英[daɪ\'vɜːsɪtɪ; dɪ-]美 [dɪˈvəsɪti]
# n.多样性；差异
return len(text)/len(set(text))
def percentage(count, total):
return 100*count/total
print(\'text3中词汇多样性指标：{}\'.format(lexical_diversity(text3)))
print(\'text4中单词a占全文的百分比：{}\'.format(percentage(text4.count(\'a\'),len(text4))))
text3中词汇多样性指标：16.050197203298673
text4中单词a占全文的百分比：1.4643016433938312
列表 = Lists
sent1 = [\'Call\', \'me\',\'Ishmael\',\'.\']
print(\'打印sent1中的内容：{}\'.format(sent1))
print(\'打印sent1中内容的长度：{}\'.format(len(sent1)))
print(\'sent1中词汇多样性指标：{}\'.format(lexical_diversity(sent1)))
打印sent1中的内容：[\'Call\', \'me\', \'Ishmael\', \'.\']
打印sent1中内容的长度：4
sent1中词汇多样性指标：1.0
sent1,sent2,sent3,sent4 # 这是内部定义好的列表
([\'Call\', \'me\', \'Ishmael\', \'.\'],
[\'The\',
\'family\',
\'of\',
\'Dashwood\',
\'had\',
\'long\',
\'been\',
\'settled\',
\'in\',
\'Sussex\',
\'.\'],
[\'In\',
\'the\',
\'beginning\',
\'God\',
\'created\',
\'the\',
\'heaven\',
\'and\',
\'the\',
\'earth\',
\'.\'],
[\'Fellow\',
\'-\',
\'Citizens\',
\'of\',
\'the\',
\'Senate\',
\'and\',
\'of\',
\'the\',
\'House\',
\'of\',
\'Representatives\',
\':\'])
sent4+sent1
[\'Fellow\',
\'-\',
\'Citizens\',
\'of\',
\'the\',
\'Senate\',
\'and\',
\'of\',
\'the\',
\'House\',
\'of\',
\'Representatives\',
\':\',
\'Call\',
\'me\',
\'Ishmael\',
\'.\']
sent1.append(\'Some\')
[\'Call\', \'me\', \'Ishmael\', \'.\', \'Some\', \'Some\', \'Some\', \'Some\']
列表索引
type(text4)
nltk.text.Text
text4[173]
\'awaken\'
text4.index(\'awaken\')
173
text5[16715:16735]
[\'U86\',
\'thats\',
\'why\',
\'something\',
\'like\',
\'gamefly\',
\'is\',
\'so\',
\'good\',
\'because\',
\'you\',
\'can\',
\'actually\',
\'play\',
\'a\',
\'full\',
\'game\',
\'without\',
\'buying\',
\'it\']
text6[1600:1625]
[\'We\',
"\'",
\'re\',
\'an\',
\'anarcho\',
\'-\',
\'syndicalist\',
\'commune\',
\'.\',
\'We\',
\'take\',
\'it\',
\'in\',
\'turns\',
\'to\',
\'act\',
\'as\',
\'a\',
\'sort\',
\'of\',
\'executive\',
\'officer\',
\'for\',
\'the\',
\'week\']
变量
sent1 = [\'Call\',\'me\',\'Ishmael\',\'.\']
my_sent = [\'Bravely\',\'bold\',\'Sir\',\'Robin\',\',\',\'rode\',\'forth\',\'from\',\'Camelot\',\'.\']
noun_phrase = my_sent[1:4]
print(\'打印切片后的列表：noun_phrase-》{}\'.format(noun_phrase))
wOrDs = sorted(noun_phrase)
print(\'打印排序后的列表：wOrDs-》{}\'.format(wOrDs))
打印切片后的列表：noun_phrase-》[\'bold\', \'Sir\', \'Robin\']
打印排序后的列表：wOrDs-》[\'Robin\', \'Sir\', \'bold\']
字符串
name = \'bright\'
print(\'打印name中的第一个字母：{}\'.format(name[0]))
print(name[:4])
print(name*2)
print(name + \'!\')
打印name中的第一个字母：b
brig
brightbright
bright!
\' \'.join([\'Monty\', \'Python\'])
\'Monty Python\'
\'Monty Python\'.split()
[\'Monty\', \'Python\']
saying = [\'After\',\'all\',\'is\',\'said\',\'and\',\'done\',\'more\',\'is\',\'said\',\'than\',\'done\']
tokens = set(saying)
tokens = sorted(tokens)
tokens[-2:]
[\'said\', \'than\']
fdist1 = FreqDist(text1)
vocabulary1 = fdist1.keys()
type(vocabulary1)
dict_keys
fdist1.plot(50, cumulative=True)
#Cumulative frequency plot for the 50 most frequently used words in Moby Dick, which
#account for nearly half of the tokens.
fdist1.hapaxes() #the words that occur once only
[\'Herman\',
\'Melville\',
\']\',
\'ETYMOLOGY\',
\'Late\',
\'Consumptive\',
\'School\',
\'threadbare\',
\'lexicons\',
\'mockingly\',
\'flags\',
\'mortality\',
\'signification\',
\'HACKLUYT\',
\'Sw\',
\'HVAL\',
\'roundness\',
\'Dut\',
\'Ger\',
\'WALLEN\',
\'WALW\',
\'IAN\',
\'RICHARDSON\',
\'KETOS\',
\'GREEK\',
\'CETUS\',
\'LATIN\',
\'WHOEL\',
\'ANGLO\',
\'SAXON\',
\'WAL\',
\'HWAL\',
\'SWEDISH\',
\'ICELANDIC\',
\'BALEINE\',
\'BALLENA\',
\'FEGEE\',
\'ERROMANGOAN\',
\'Librarian\',
\'painstaking\',
\'burrower\',
\'grub\',
\'Vaticans\',
\'stalls\',
\'higgledy\',
\'piggledy\',
\'gospel\',
\'promiscuously\',
\'commentator\',
\'belongest\',
\'sallow\',
\'Pale\',
\'Sherry\',
\'loves\',
\'bluntly\',
\'Subs\',
\'thankless\',
\'Hampton\',
\'Court\',
\'hie\',
\'refugees\',
\'pampered\',
\'Michael\',
\'Raphael\',
\'unsplinterable\',
\'GENESIS\',
\'JOB\',
\'JONAH\',
\'punish\',
\'ISAIAH\',
\'soever\',
\'cometh\',
\'incontinently\',
\'perisheth\',
\'PLUTARCH\',
\'MORALS\',
\'breedeth\',
\'Whirlpooles\',
\'Balaene\',
\'arpens\',
\'PLINY\',
\'Scarcely\',
\'TOOKE\',
\'LUCIAN\',
\'TRUE\',
\'catched\',
\'OCTHER\',
\'VERBAL\',
\'TAKEN\',
\'MOUTH\',
\'ALFRED\',
\'890\',
\'gudgeon\',
\'retires\',
\'MONTAIGNE\',
\'APOLOGY\',
\'RAIMOND\',
\'SEBOND\',
\'Nick\',
\'RABELAIS\',
\'cartloads\',
\'STOWE\',
\'ANNALS\',
\'LORD\',
\'BACON\',
\'Touching\',
\'ork\',
\'DEATH\',
\'sovereignest\',
\'bruise\',
\'HAMLET\',
\'leach\',
\'Mote\',
\'availle\',
\'returne\',
\'againe\',
\'worker\',
\'Dinting\',
\'paine\',
\'thro\',
\'maine\',
\'FAERIE\',
\'Immense\',
\'til\',
\'DAVENANT\',
\'PREFACE\',
\'GONDIBERT\',
\'spermacetti\',
\'Hosmannus\',
\'Nescio\',
\'VIDE\',
\'Spencer\',
\'Talus\',
\'flail\',
\'threatens\',
\'jav\',
\'lins\',
\'WALLER\',
\'SUMMER\',
\'ISLANDS\',
\'Commonwealth\',
\'Civitas\',
\'OPENING\',
\'SENTENCE\',
\'HOBBES\',
\'LEVIATHAN\',
\'Silly\',
\'Mansoul\',
\'chewing\',
\'sprat\',
\'PILGRIM\',
\'PROGRESS\',
\'Created\',
\'PARADISE\',
\'LOST\',
\'---"\',
\'Hugest\',
\'Stretched\',
\'Draws\',
\'FULLLER\',
\'PROFANE\',
\'HOLY\',
\'STATE\',
\'DRYDEN\',
\'ANNUS\',
\'MIRABILIS\',
\'aground\',
\'EDGE\',
\'TEN\',
\'SPITZBERGEN\',
\'PURCHAS\',
\'wantonness\',
\'fuzzing\',
\'vents\',
\'HERBERT\',
\'INTO\',
\'ASIA\',
\'AFRICA\',
\'SCHOUTEN\',
\'SIXTH\',
\'CIRCUMNAVIGATION\',
\'Elbe\',
\'ducat\',
\'herrings\',
\'GREENLAND\',
\'Several\',
\'Fife\',
\'Anno\',
\'1652\',
\'Pitferren\',
\'SIBBALD\',
\'FIFE\',
\'KINROSS\',
\'Myself\',
\'Sperma\',
\'ceti\',
\'fierceness\',
\'RICHARD\',
\'STRAFFORD\',
\'LETTER\',
\'BERMUDAS\',
\'PHIL\',
\'TRANS\',
\'1668\',
\'PRIMER\',
\'COWLEY\',
\'1729\',
\'"...\',
\'frequendy\',
\'insupportable\',
\'disorder\',
\'ULLOA\',
\'SOUTH\',
\'AMERICA\',
\'sylphs\',
\'petticoat\',
\'Oft\',
\'Tho\',
\'RAPE\',
\'LOCK\',
\'NAT\',
\'wales\',
\'JOHNSON\',
\'COOK\',
\'dung\',
\'lime\',
\'juniper\',
\'UNO\',
\'VON\',
\'TROIL\',
\'LETTERS\',
\'BANKS\',
\'SOLANDER\',
\'1772\',
\'Nantuckois\',
\'JEFFERSON\',
\'MEMORIAL\',
\'MINISTER\',
\'REFERENCE\',
\'PARLIAMENT\',
\'SOMEWHERE\',
\'guarding\',
\'protecting\',
\'robbers\',
\'BLACKSTONE\',
\'Rodmond\',
\'suspends\',
\'attends\',
\'FALCONER\',
\'Bright\',
\'roofs\',
\'domes\',
\'rockets\',
\'Around\',
\'unwieldy\',
\'COWPER\',
\'VISIT\',
\'LONDON\',
\'HUNTER\',
\'DISSECTION\',
\'SMALL\',
\'SIZED\',
\'aorta\',
\'gushing\',
\'PALEY\',
\'THEOLOGY\',
\'mammiferous\',
\'hind\',
\'BARON\',
\'CUVIER\',
\'COLNETT\',
\'PURPOSE\',
\'EXTENDING\',
\'SPERMACETI\',
\'Floundered\',
\'chace\',
\'peopling\',
\'Gather\',
\'Led\',
\'instincts\',
\'trackless\',
\'Assaulted\',
\'voracious\',
\'spiral\',
\'MONTGOMERY\',
\'WORLD\',
\'FLOOD\',
\'Paean\',
\'fatter\',
\'Flounders\',
\'CHARLES\',
\'LAMB\',
\'TRIUMPH\',
\'1690\',
\'OBED\',
\'Susan\',
\'HAWTHORNE\',
\'TWICE\',
\'bespeak\',
\'raal\',
\'COOPER\',
\'PILOT\',
\'Berlin\',
\'Gazette\',
\'ECKERMANN\',
\'CONVERSATIONS\',
\'GOETHE\',
\'ESSEX\',
\'WAS\',
\'ATTACKED\',
\'FINALLY\',
\'DESTROYED\',
\'OWEN\',
\'CHACE\',
\'FIRST\',
\'SAID\',
\'VESSEL\',
\'YORK\',
\'1821\',
\'piping\',
\'dimmed\',
\'phospher\',
\'ELIZABETH\',
\'OAKES\',
\'SMITH\',
\'amounted\',
\'440\',
\'SCORESBY\',
\'Mad\',
\'agonies\',
\'endures\',
\'infuriated\',
\'rears\',
\'snaps\',
\'propelled\',
\'observers\',
\'opportunities\',
\'habitudes\',
\'BEALE\',
\'offensively\',
\'artful\',
\'mischievous\',
\'FREDERICK\',
\'DEBELL\',
\'1840\',
\'October\',
\'Raise\',
\'ay\',
\'THAR\',
\'bowes\',
\'os\',
\'ROSS\',
\'ETCHINGS\',
\'CRUIZE\',
\'1846\',
\'Globe\',
\'transactions\',
\'relate\',
\'HUSSEY\',
\'SURVIVORS\',
\'parried\',
\'MISSIONARY\',
\'JOURNAL\',
\'TYERMAN\',
\'boldest\',
\'persevering\',
\'REPORT\',
\'DANIEL\',
\'SPEECH\',
\'SENATE\',
\'APPLICATION\',
\'ERECTION\',
\'BREAKWATER\',
\'CAPTORS\',
\'WHALEMAN\',
\'ADVENTURES\',
\'BIOGRAPHY\',
\'GATHERED\',
\'HOMEWARD\',
\'COMMODORE\',
\'PREBLE\',
\'REV\',
\'CHEEVER\',
\'MUTINEER\',
\'BROTHER\',
\'ANOTHER\',
\'MCCULLOCH\',
\'COMMERCIAL\',
\'reciprocal\',
\'clews\',
\'SOMETHING\',
\'UNPUBLISHED\',
\'CURRENTS\',
\'Pedestrians\',
\'recollect\',
\'gateways\',
\'VOYAGER\',
\'ARCTIC\',
\'NEWSPAPER\',
\'TAKING\',
\'RETAKING\',
\'HOBOMACK\',
\'MIRIAM\',
\'FISHERMAN\',
\'appliance\',
\'RIBS\',
\'TRUCKS\',
\'Terra\',
\'Del\',
\'Fuego\',
\'DARWIN\',
\'NATURALIST\',
";--\'",
\'!\\\'"\',
\'WHARTON\',
\'Loomings\',
\'spleen\',
\'regulating\',
\'circulation\',
\'Whenever\',
\'drizzly\',
\'hypos\',
\'philosophical\',
\'Cato\',
\'Manhattoes\',
\'reefs\',
\'downtown\',
\'gazers\',
\'Circumambulate\',
\'Corlears\',
\'Coenties\',
\'Slip\',
\'Whitehall\',
\'Posted\',
\'sentinels\',
\'spiles\',
\'pier\',
\'lath\',
\'counters\',
\'desks\',
\'loitering\',
\'shady\',
\'Inlanders\',
\'lanes\',
\'alleys\',
\'attract\',
\'dale\',
\'dreamiest\',
\'shadiest\',
\'quietest\',
\'enchanting\',
\'Saco\',
\'crucifix\',
\'Deep\',
\'mazy\',
\'Tiger\',
\'Tennessee\',
\'Rockaway\',
\'Persians\',
\'deity\',
\'Narcissus\',
\'ungraspable\',
\'hazy\',
\'quarrelsome\',
\'offices\',
\'abominate\',
\'toils\',
\'trials\',
\'barques\',
\'schooners\',
\'broiling\',
\'buttered\',
\'judgmatically\',
\'peppered\',
\'reverentially\',
\'idolatrous\',
\'dotings\',
\'ibis\',
\'roasted\',
\'bake\',
\'plumb\',
\'Van\',
\'Rensselaers\',
\'Randolphs\',
\'Hardicanutes\',
\'lording\',
\'tallest\',
\'decoction\',
\'Seneca\',
\'Stoics\',
\'Testament\',
\'promptly\',
\'rub\',
\'infliction\',
\'BEING\',
\'PAID\',
\'urbane\',
\'ills\',
\'monied\',
\'consign\',
\'prevalent\',
\'violate\',
\'Pythagorean\',
\'commonalty\',
\'police\',
\'surveillance\',
\'programme\',
\'solo\',
\'CONTESTED\',
\'ELECTION\',
\'PRESIDENCY\',
\'UNITED\',
\'STATES\',
\'ISHMAEL\',
\'BLOODY\',
\'AFFGHANISTAN\',
\'managers\',
\'genteel\',
\'comedies\',
\'farces\',
\'cunningly\',
\'disguises\',
\'cajoling\',
\'unbiased\',
\'freewill\',
\'discriminating\',
\'overwhelming\',
\'undeliverable\',
\'itch\',
\'forbidden\',
\'ignoring\',
\'lodges\',
\'Carpet\',
\'Bag\',
\'Manhatto\',
\'candidates\',
\'penalties\',
\'Tyre\',
\'Carthage\',
\'imported\',
\'cobblestones\',
\'bitingly\',
\'shouldering\',
\'price\',
\'fervent\',
\'asphaltic\',
\'pavement\',
\'flinty\',
\'projections\',
\'soles\',
\'Too\',
\'cheapest\',
\'cheeriest\',
\'invitingly\',
\'particles\',
\'peer\',
\'Angel\',
\'Doom\',
\'wailing\',
\'gnashing\',
\'Wretched\',
\'entertainment\',
\'Moving\',
\'emigrant\',
\'poverty\',
\'creak\',
\'lodgings\',
\'zephyr\',
\'hob\',
\'toasting\',
\'observest\',
\'sashless\',
\'glazier\',
\'reasonest\',
\'chinks\',
\'crannies\',
\'lint\',
\'chattering\',
\'shiverings\',
\'cob\',
\'redder\',
\'Orion\',
\'glitters\',
\'conservatories\',
\'president\',
\'temperance\',
\'blubbering\',
\'straggling\',
\'wainscots\',
\'reminding\',
\'oilpainting\',
\'besmoked\',
\'defaced\',
\'unequal\',
\'crosslights\',
\'hags\',
\'delineate\',
\'bewitched\',
\'ponderings\',
\'boggy\',
\'soggy\',
\'squitchy\',
\'froze\',
\'heath\',
\'icebound\',
\'represents\',
\'Horner\',
\'foundered\',
\'clubs\',
\'harvesting\',
\'hacking\',
\'horrifying\',
\'Mixed\',
\'Nathan\',
\'Swain\',
\'corkscrew\',
\'Blanco\',
\'sojourning\',
\'fireplaces\',
\'duskier\',
\'cockpits\',
\'rarities\',
\'Projecting\',
\'Within\',
\'shelves\',
\'flasks\',
\'bustles\',
\'deliriums\',
\'Abominable\',
\'tumblers\',
\'cylinders\',
\'goggling\',
\'deceitfully\',
\'tapered\',
\'Parallel\',
\'pecked\',
\'footpads\',
\'Fill\',
\'shilling\',
\'examining\',
\'SKRIMSHANDER\',
\'accommodated\',
\'unoccupied\',
\'haint\',
\'pose\',
\'whalin\',
\'decidedly\',
\'objectionable\',
\'wander\',
\'Battery\',
\'ruminating\',
\'adorning\',
\'potatoes\',
\'sartainty\',
\'diabolically\',
\'steaks\',
\'undress\',
\'looker\',
\'rioting\',
\'Grampus\',
\'seed\',
\'Feegees\',
\'tramping\',
\'Enveloped\',
\'bedarned\',
\'eruption\',
\'officiating\',
\'brimmers\',
\'complained\',
\'potion\',
\'colds\',
\'catarrhs\',
\'liquor\',
\'arrantest\',
\'topers\',
\'obstreperously\',
\'aloof\',
\'desirous\',
\'hilarity\',
\'coffer\',
\'Southerner\',
\'mountaineers\',
\'Alleghanian\',
\'missed\',
\'supernaturally\',
\'congratulate\',
\'multiply\',
\'bachelor\',
\'abominated\',
\'tidiest\',
\'bedwards\',
\'shan\',
\'tablecloth\',
\'Skrimshander\',
\'bump\',
\'spraining\',
\'eider\',
\'yoking\',
\'rickety\',
\'whirlwinds\',
\'knockings\',
\'dismissed\',
\'popped\',
\'cherishing\',
\'chuckled\',
\'chuckle\',
\'mightily\',
\'catches\',
\'bamboozingly\',
\'overstocked\',
\'toothpick\',
\'rayther\',
\'BROWN\',
\'slanderin\',
\'farrago\',
\'BROKE\',
\'Sartain\',
\'Mt\',
\'Hecla\',
\'persist\',
\'mystifying\',
\'unsay\',
\'criminal\',
\'Wall\',
\'purty\',
\'sarmon\',
\'rips\',
\'tellin\',
\'bought\',
\'balmed\',
\'curios\',
\'sellin\',
\'inions\',
\'fooling\',
\'idolators\',
\'Depend\',
\'reg\',
\'lar\',
\'spliced\',
\'Johnny\',
\'sprawling\',
\'Arter\',
\'glim\',
\'jiffy\',
\'irresolute\',
\'vum\',
\'WON\',
\'Folding\',
\'scrutiny\',
\'porcupine\',
\'moccasin\',
\'ponchos\',
\'parade\',
\'rainy\',
\'remembering\',
\'commended\',
\'cobs\',
\'Nod\',
\'footfall\',
\'unlacing\',
\'blackish\',
\'plasters\',
\'inkling\',
\'Placing\',
\'crammed\',
\'scalp\',
\'mildewed\',
\'Ignorance\',
\'parent\',
\'nonplussed\',
\'undressing\',
\'checkered\',
\'Thirty\',
\'frogs\',
\'quaked\',
\'wrapall\',
\'dreadnaught\',
\'fumbled\',
\'Remembering\',
\'manikin\',
\'tenpin\',
\'andirons\',
\'jambs\',
\'bricks\',
\'appropriate\',
\'applying\',
\'hastier\',
\'withdrawals\',
\'antics\',
\'devotee\',
\'extinguishing\',
\'unceremoniously\',
\'bagged\',
\'sportsman\',
\'woodcock\',
\'uncomfortableness\',
\'deliberating\',
\'puffed\',
\'sang\',
\'Stammering\',
\'conjured\',
\'responses\',
\'debel\',
\'flourishing\',
\'Angels\',
\'flourishings\',
\'peddlin\',
\'sleepe\',
\'grunted\',
\'gettee\',
\'motioning\',
\'comely\',
\'insured\',
\'Counterpane\',
\'parti\',
\'triangles\',
\'interminable\',
\'caper\',
\'supperless\',
\'21st\',
\'hemisphere\',
\'sigh\',
\'Sixteen\',
\'ached\',
\'coaches\',
\'stockinged\',
\'slippering\',
\'misbehaviour\',
\'unendurable\',
\'stepmothers\',
\'misfortunes\',
\'steeped\',
\'shudderingly\',
\'confounding\',
\'soberly\',
\'recurred\',
\'predicament\',
\'unlock\',
\'bridegroom\',
\'clasp\',
\'hugged\',
\'rouse\',
\'snore\',
\'scratch\',
\'Throwing\',
\'expostulations\',
\'unbecomingness\',
\'matrimonial\',
\'dawning\',
\'overture\',
\'innate\',
\'compliment\',
\'civility\',
\'rudeness\',
\'toilette\',
\'dressing\',
\'donning\',
\'gaspings\',
\'booting\',
\'caterpillar\',
\'outlandishness\',
\'manners\',
\'education\',
\'undergraduate\',
\'dreamt\',
\'cowhide\',
\'pinched\',
\'curtains\',
\'indecorous\',
\'contented\',
\'restricting\',
\'donned\',
\'lathering\',
\'unsheathes\',
\'whets\',
\'Rogers\',
\'cutlery\',
\'Afterwards\',
\'baton\',
\'Breakfast\',
\'pleasantly\',
\'bountifully\',
\'laughable\',
\'bosky\',
\'unshorn\',
\'gowns\',
\'toasted\',
\'lingers\',
\'tarried\',
\'barred\',
\'Grub\',
\'Park\',
\'assurance\',
\'polish\',
\'occasioned\',
\'embarrassed\',
\'bashfulness\',
\'duelled\',
\'winking\',
\'tastes\',
\'sheepishly\',
\'bashful\',
\'icicle\',
\'admirer\',
\'cordially\',
\'grappling\',
\'genteelly\',
\'eschewed\',
\'undivided\',
\'6\',
\'circulating\',
\'nondescripts\',
\'Chestnut\',
\'jostle\',
\'Regent\',
\'Lascars\',
\'Bombay\',
\'Apollo\',
\'Feegeeans\',
\'Tongatobooarrs\',
\'Erromanggoans\',
\'Pannangians\',
\'Brighggians\',
\'weekly\',
\'Vermonters\',
\'stalwart\',
\'frames\',
\'felled\',
\'strutting\',
\'wester\',
\'bombazine\',
\'cloak\',
\'mow\',
\'gloves\',
\'joins\',
\'outfit\',
\'waistcoats\',
\'Hay\',
\'Seed\',
\'tract\',
\'dearest\',
\'pave\',
\'eggs\',
\'patrician\',
\'parks\',
\'scraggy\',
\'scoria\',
\'Herr\',
\'dowers\',
\'nieces\',
\'reservoirs\',
\'maples\',
\'bountiful\',
\'proffer\',
\'passer\',
\'cones\',
\'blossoms\',
\'superinduced\',
\'carnation\',
\'Salem\',
\'sweethearts\',
\'Puritanic\',
\'Whaleman\',
\'Wrapping\',
\'Each\',
\'quote\',
\'TALBOT\',
\'Near\',
\'Desolation\',
\'1st\',
\'SISTER\',
\'ROBERT\',
\'WILLIS\',
\'ELLERY\',
\'NATHAN\',
\'COLEMAN\',
\'WALTER\',
\'CANNY\',
\'SETH\',
\'GLEIG\',
\'Forming\',
\'ELIZA\',
\'31st\',
\'MARBLE\',
\'SHIPMATES\',
\'EZEKIEL\',
\'HARDY\',
\'AUGUST\',
\'3d\',
\'1833\',
\'WIDOW\',
\'Shaking\',
\'glazed\',
\'Affected\',
\'relatives\',
\'unhealing\',
\'sympathetically\',
\'wounds\',
\'bleed\',
\'blanks\',
...]
单词的精细选择
- the set of all w such that w is an element of V (the vocabulary) and w has property P
{w|w \\(\\in\\) V and P(w)} - The corresponding Python expression is given:
[w for w in V if p(w)]
V = set(text1)
long_words = [w for w in V if len(w)>15]
sorted(long_words)
[\'CIRCUMNAVIGATION\',
\'Physiognomically\',
\'apprehensiveness\',
\'cannibalistically\',
\'characteristically\',
\'circumnavigating\',
\'circumnavigation\',
\'circumnavigations\',
\'comprehensiveness\',
\'hermaphroditical\',
\'indiscriminately\',
\'indispensableness\',
\'irresistibleness\',
\'physiognomically\',
\'preternaturalness\',
\'responsibilities\',
\'simultaneousness\',
\'subterraneousness\',
\'supernaturalness\',
\'superstitiousness\',
\'uncomfortableness\',
\'uncompromisedness\',
\'undiscriminating\',
\'uninterpenetratingly\']
本文选自《Natural Language Processing with Python》