分类和标注词汇

Posted 2020-12-31 nxf-rabbit75

1.词性标注器 parts-of-speech 或 POS tagger  nltk自带英文标注器

import nltk

text = nltk.word_tokenize("And now for something completely different")#list

print(nltk.pos_tag(text))

print(nltk.help.upenn_tagset(‘RB‘))#标记的文档

text1 = nltk.Text(word.lower() for word in nltk.corpus.brown.words())

print(type(text1))#<class ‘nltk.text.Text‘>

text1.similar(‘bought‘)

 

2.标注语料库 

3.自动标注

4.N-gram标注

5.基于转换的标注：Brill标注器