使用Lucene 7 OpenNLP查询词性标签

Question

为了娱乐和学习，我正在尝试使用OpenNLP和Lucene 7.4构建一个词性（POS）标记器。目标是，一旦索引，我实际上可以搜索一系列POS标签并找到所有与序列匹配的句子。我已经获得了索引部分，但我仍然坚持查询部分。我知道SolR可能有一些功能，我已经检查了代码（毕竟不是那么自我解释）。但我的目标是在Lucene 7中理解和实现，而不是在SolR中，因为我希望独立于任何搜索引擎。

想法输入句子1：快速的棕色狐狸跳过懒狗。应用Lucene OpenNLP标记器导致：[[]] [快] [棕色] [狐狸] [跳跃] [结束] [] [懒惰] [狗] [。]接下来，应用Lucene OpenNLP POS标记导致：[DT] [JJ] [JJ] [NN] [VBD] [IN] [DT] [JJ] [NNS] [。]

输入句子2：给我，宝贝！ Applied Lucene OpenNLP tokenizer导致：[Give] [it] [to] [me] [，] [baby] [！]接下来，应用Lucene OpenNLP POS标记会导致：[VB] [PRP] [TO] [PRP] [，] [UH] [。]

查询：JJ NN VBD匹配句子1的一部分，因此应返回句子1。（此时我只对完全匹配感兴趣，即让我们将部分匹配，通配符等放在一边）

索引首先，我创建了自己的类com.example.OpenNLPAnalyzer：

public class OpenNLPAnalyzer extends Analyzer {
  protected TokenStreamComponents createComponents(String fieldName) {
    try {

        ResourceLoader resourceLoader = new ClasspathResourceLoader(ClassLoader.getSystemClassLoader());


        TokenizerModel tokenizerModel = OpenNLPOpsFactory.getTokenizerModel("en-token.bin", resourceLoader);
        NLPTokenizerOp tokenizerOp = new NLPTokenizerOp(tokenizerModel);


        SentenceModel sentenceModel = OpenNLPOpsFactory.getSentenceModel("en-sent.bin", resourceLoader);
        NLPSentenceDetectorOp sentenceDetectorOp = new NLPSentenceDetectorOp(sentenceModel);

        Tokenizer source = new OpenNLPTokenizer(
                AttributeFactory.DEFAULT_ATTRIBUTE_FACTORY, sentenceDetectorOp, tokenizerOp);

        POSModel posModel = OpenNLPOpsFactory.getPOSTaggerModel("en-pos-maxent.bin", resourceLoader);
        NLPPOSTaggerOp posTaggerOp = new NLPPOSTaggerOp(posModel);

        // Perhaps we should also use a lower-case filter here?

        TokenFilter posFilter = new OpenNLPPOSFilter(source, posTaggerOp);

        // Very important: Tokens are not indexed, we need a store them as payloads otherwise we cannot search on them
        TypeAsPayloadTokenFilter payloadFilter = new TypeAsPayloadTokenFilter(posFilter);

        return new TokenStreamComponents(source, payloadFilter);
    }
    catch (IOException e) {
        throw new RuntimeException(e.getMessage());
    }              

}

请注意，我们使用的是围绕OpenNLPPOSFilter的TypeAsPayloadTokenFilter。这意味着，我们的POS标签将被索引为有效载荷，而我们的查询 - 无论它看起来如何 - 也必须搜索有效载荷。

查询这是我被困的地方。我不知道如何查询有效负载，无论我尝试什么都行不通。请注意，我使用的是Lucene 7，似乎在旧版本中查询有效负载已多次更改。文档非常稀缺。现在还不清楚现在要查询的正确字段名称 - 是“单词”还是“类型”还是其他什么？例如，我尝试了此代码，但不返回任何搜索结果：

    // Step 1: Indexing
    final String body = "The quick brown fox jumped over the lazy dogs.";
    Directory index = new RAMDirectory();
    OpenNLPAnalyzer analyzer = new OpenNLPAnalyzer();
    IndexWriterConfig indexWriterConfig = new IndexWriterConfig(analyzer);
    IndexWriter writer = new IndexWriter(index, indexWriterConfig);
    Document document = new Document();
    document.add(new TextField("body", body, Field.Store.YES));
    writer.addDocument(document);
    writer.close();


    // Step 2: Querying
    final int topN = 10;
    DirectoryReader reader = DirectoryReader.open(index);
    IndexSearcher searcher = new IndexSearcher(reader);

    final String fieldName = "body"; // What is the correct field name here? "body", or "type", or "word" or anything else?
    final String queryText = "JJ";
    Term term = new Term(fieldName, queryText);
    SpanQuery match = new SpanTermQuery(term);
    BytesRef pay = new BytesRef("type"); // Don't understand what to put here as an argument
    SpanPayloadCheckQuery query = new SpanPayloadCheckQuery(match, Collections.singletonList(pay));

    System.out.println(query.toString());

    TopDocs topDocs = searcher.search(query, topN);

这里非常感谢任何帮助。