elasticsearch实战三部曲之三：搜索操作

Posted 2022-08-02 程序员欣宸

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了elasticsearch实战三部曲之三：搜索操作相关的知识，希望对你有一定的参考价值。

欢迎访问我的GitHub

本文是《elasticsearch实战三部曲》的终篇，作为elasticsearch的核心功能，搜索的重要性不言而喻，今天的实战都会围绕搜索展开；

环境信息

本次实战用到的elasticsearch版本是6.5.4，安装在Ubuntu 16.04.5 LTS，客户端工具是postman6.6.1；
如果您需要搭建elasticsearch环境，请参考《Linux环境快速搭建elasticsearch6.5.4集群和Head插件》；

基本情况介绍

本次实战的elasticsearch环境以及搭建完毕，是由两个机器搭建的集群，并且elasticsearch-head也搭建完成：
一号机器，IP地址：192.168.119.152；
二号机器：IP地址：192.168.119.153；
elasticsearch-head安装在一号机器，访问地址：http://192.168.119.152:9100
已经建立了索引englishbooks，对应的数据如下所示，请用批量命令导入到elasticsearch：

"index": "_index": "englishbooks", "_type": "IT", "_id": "1" 
"id":"1","title":"Deep Learning","language":"python","author":"Yoshua Bengio","price":549.00,"publish_time":"2016-11-18","description":"written by three experts in the field, deep learning is the only comprehensive book on the subject."
"index": "_index": "englishbooks", "_type": "IT", "_id": "2" 
"id":"2","title":"Compilers","language":"c","author":"Alfred V.Aho","price":62.50,"publish_time":"2011-01-01","description":"In the time since the 1986 edition of this book, the world of compiler designhas changed significantly."
"index": "_index": "englishbooks", "_type": "IT", "_id": "3" 
"id":"3","title":"Core Java","language":"java","author":"Horstmann","price":85.90,"publish_time":"2016-06-01","description":"The book is aimed at experienced programmers who want to learn how to write useful Java applications and applets. "
"index": "_index": "englishbooks", "_type": "IT", "_id": "4" 
"id":"4","title":"Thinking in Java","language":"java","author":"Bruce Eckel","price":70.10,"publish_time":"2015-07-06","description":"Thinking in Java should be read cover to cover by every Java programmer, then kept close at hand for frequent reference. The exercises are challenging, and the chapter on Collections is superb!"
"index": "_index": "englishbooks", "_type": "IT", "_id": "5" 
"id":"5","title":"The Go Programming Language","language":"go","author":"Alan A.A.Donovan","price":63.90,"publish_time":"2016-01-01","description":"A declarations lexical block determines its scope， which may be large or small. The declarations of built—in types， functions， and constants like int， len， and true are in the universe block and can be referred to throughout the entire program."

相关的文档是批量导入的，关于文档数据和批量操作的细节请参考《elasticsearch实战三部曲之二：文档操作》；
books索引的文档内容在head中展示如下图：

数据格式说明

为了便于和读者沟通，我们来约定一下如何在文章中表达请求和响应的信息：
假设通过Postman工具向服务器发送一个PUT类型的请求，地址是：http://192.168.119.152:9200/test001/article/1
请求的内容是JSON格式的，内容如下：


    “id”:1,
    "title":"标题a",
    "posttime":"2019-01-12",
    "content":"一起来熟悉文档相关的操作"

对于上面的请求，我在文章中就以如下格式描述：
```
PUT test001/article/1
```

“id”:1,
"title":"标题a",
"posttime":"2019-01-12",
"content":"一起来熟悉文档相关的操作"

- 读者您看到上述内容，就可以在postman中发起PUT请求，地址是"test001/article/1"前面加上您的服务器地址，内容是上面的JSON；

### 本文中的文档内容暂不涉及中文
- 文中数据都是英文的，避免在因分词器的分词问题导致搜索不到对应的中文结果，分词器相关的知识会在另一篇文章中详细介绍；

### 查看所有数据
```shell
GET englishbooks/_search


    "query":
        "match_all":

上述查询返回索引books的所有记录，并且文档得分收是1；
您可以将请求的整个JSON删除，只用books/_search这个URL来试试，也能得到所有数据，这是match_all的简写；

数字字段的精确匹配

查询价格等于549的记录：

GET englishbooks/_search


    "query":
        "constant_score":
            "filter":
                "term":"price":549

得到结果：


    "took": 4,
    "timed_out": false,
    "_shards": 
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    ,
    "hits": 
        "total": 1,
        "max_score": 1,
        "hits": [
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "1",
                "_score": 1,
                "_source": 
                    "id": "1",
                    "title": "Deep Learning",
                    "language": "python",
                    "author": "Yoshua Bengio",
                    "price": 549,
                    "publish_time": "2016-11-18",
                    "description": "written by three experts in the field, deep learning is the only comprehensive book on the subject."
                
            
        ]

请求参数中使用了constant_score 后，查询将以非评分模式来执行 term，并以一作为统一评分；

查看分词效果

text类型的字段会被分词后构建倒排索引，来看看title字段的值为"Core Java"时的分词效果：

GET englishbooks/_analyze


    "field":"title",
    "text":"Core Java"

响应如下所示，"Core Java"被分"core"和"java"两个词，也就是说我们以词项"core"或"java"搜索title字段都能收到对应文档：


    "tokens": [
        
            "token": "core",
            "start_offset": 0,
            "end_offset": 4,
            "type": "<ALPHANUM>",
            "position": 0
        ,
        
            "token": "java",
            "start_offset": 5,
            "end_offset": 9,
            "type": "<ALPHANUM>",
            "position": 1
        
    ]

需要注意的是分词后的结果都是小写，这是分词器的处理结果；

词项查询（term query）

前面我们查看分词效果发现"Core Java"被分"core"和"java"两个词，现在就以"java"为关键词搜索一下试试：

GET englishbooks/_search


    "query":
        "term":"title":"java"

结果如下，title中有java关键词的两个文档都被搜到：


    "took": 4,
    "timed_out": false,
    "_shards": 
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    ,
    "hits": 
        "total": 2,
        "max_score": 0.5754429,
        "hits": [
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "4",
                "_score": 0.5754429,
                "_source": 
                    "id": "4",
                    "title": "Thinking in Java",
                    "language": "java",
                    "author": "Bruce Eckel",
                    "price": 70.1,
                    "publish_time": "2015-07-06",
                    "description": "Thinking in Java should be read cover to cover by every Java programmer, then kept close at hand for frequent reference. The exercises are challenging, and the chapter on Collections is superb!"
                
            ,
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "3",
                "_score": 0.2876821,
                "_source": 
                    "id": "3",
                    "title": "Core Java",
                    "language": "java",
                    "author": "Horstmann",
                    "price": 85.9,
                    "publish_time": "2016-06-01",
                    "description": "The book is aimed at experienced programmers who want to learn how to write useful Java applications and applets. "
                
            
        ]

分词查询（match query）

term query的特点是将输入的内容作为一个词项来用，例如以下的查询是没有结果的：

GET englishbooks/_search


    "query":
        "term":"title":"core java"

上述查询没有结果的原因，是因为"core java"被当做一个词项去查询了，而title的分词结果中只有"core"、"java"这些分词过的词项，并没有一个叫做"core java"的词项，所以搜不到结果；
如果输入的查询条件"core java"也被做一次分词处理，再把处理结果"core"和"java"用来搜索，应该就能得到结果了，match query就是用来对输入条件做分词处理的，如下：

GET englishbooks/_search


    "query":
        "match":"title":"Core Java"

搜索结果如下，包含了java的两条记录都被查出来了：


    "took": 8,
    "timed_out": false,
    "_shards": 
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    ,
    "hits": 
        "total": 2,
        "max_score": 0.5754429,
        "hits": [
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "4",
                "_score": 0.5754429,
                "_source": 
                    "id": "4",
                    "title": "Thinking in Java",
                    "language": "java",
                    "author": "Bruce Eckel",
                    "price": 70.1,
                    "publish_time": "2015-07-06",
                    "description": "Thinking in Java should be read cover to cover by every Java programmer, then kept close at hand for frequent reference. The exercises are challenging, and the chapter on Collections is superb!"
                
            ,
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "3",
                "_score": 0.5753642,
                "_source": 
                    "id": "3",
                    "title": "Core Java",
                    "language": "java",
                    "author": "Horstmann",
                    "price": 85.9,
                    "publish_time": "2016-06-01",
                    "description": "The book is aimed at experienced programmers who want to learn how to write useful Java applications and applets. "
                
            
        ]

如果我们的本意是只要"Core Java"的匹配结果，上面的结果显然是不符合要求的，此时可以给查询条件加个"operator":"and"属性，就会查询匹配了所有关键词的文档，注意json的结构略有变化，以前title的属性是搜索条件，现在变成了一个json对象，里面的query属性是原来的搜索条件：

GET englishbooks/_search


    "query":
        "match":
            "title":
                "query":"Core Java",
                "operator":"and"

这次的搜索结果就是同时匹配了"core"和"java"两个词项的记录了(为什么core和java是小写？因为"Core Java"被分词后改为了小写，再去搜索的)：


    "took": 11,
    "timed_out": false,
    "_shards": 
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    ,
    "hits": 
        "total": 1,
        "max_score": 0.5753642,
        "hits": [
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "3",
                "_score": 0.5753642,
                "_source": 
                    "id": "3",
                    "title": "Core Java",
                    "language": "java",
                    "author": "Horstmann",
                    "price": 85.9,
                    "publish_time": "2016-06-01",
                    "description": "The book is aimed at experienced programmers who want to learn how to write useful Java applications and applets. "
                
            
        ]

match_phrase搜索

match_phrase搜索和前面的match搜索相似，并且有以下两个特点：
分词后的所有词项都要匹配上，也就是前面的"operator":"and"属性的效果；
分析后的词项顺序要和搜索字段的顺序一致，才能匹配上；

GET englishbooks/_search


    "query":
        "match_phrase":"title":"Core Java"

上述查询可以搜索到结果，但如果将"Core Java"改成"Java Core"就搜不到结果了，但是match query用"Java Core"是可以搜到结果的；

match_phrase_prefix搜索

match_phrase_prefix的功能和前面的match_phrase类似，不过match_phrase_prefix支持最后一个词项做前缀匹配，如下所示，"Core J"这个搜索条件用match_phrase是搜不到结果的，但是match_phrase_prefix可以，因为"J"可以作为前缀和"Java"匹配：

GET englishbooks/_search


    "query":
        "match_phrase":"title":"Core J"

multi_match搜素

multi_match是在match的基础上支持多字段搜索，以下查询就是用"1986"和"deep"这两个词项，同时搜索title和description两个字段：

GET englishbooks/_search


    "query":
        "multi_match":
            "query":"1986 deep",
            "fields":["title", "description"]

响应如下，可见title和description中含有词项"1986"或者"deep"的文档都被返回了：


"took": 4,
"timed_out": false,
"_shards": 
    "total": 5,
    "successful": 5,
    "skipped": 0,
    "failed": 0
,
"hits": 
    "total": 2,
    "max_score": 0.79237825,
    "hits": [
        
            "_index": "englishbooks",
            "_type": "IT",
            "_id": "2",
            "_score": 0.79237825,
            "_source": 
                "id": "2",
                "title": "Compilers",
                "language": "c",
                "author": "Alfred V.Aho",
                "price": 62.5,
                "publish_time": "2011-01-01",
                "description": "In the time since the 1986 edition of this book, the world of compiler designhas changed significantly."
            
        ,
        
            "_index": "englishbooks",
            "_type": "IT",
            "_id": "1",
            "_score": 0.2876821,
            "_source": 
                "id": "1",
                "title": "Deep Learning",
                "language": "python",
                "author": "Yoshua Bengio",
                "price": 549,
                "publish_time": "2016-11-18",
                "description": "written by three experts in the field, deep learning is the only comprehensive book on the subject."
            
        
    ]

terms query

terms是term查询的升级，用来查询多个词项：

GET englishbooks/_search


    "query":
        "terms":
            "title":["deep", "core"]

响应如下，title中含有deep和core的文档都被查到：


    "took": 5,
    "timed_out": false,
    "_shards": 
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    ,
    "hits": 
        "total": 2,
        "max_score": 1,
        "hits": [
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "1",
                "_score": 1,
                "_source": 
                    "id": "1",
                    "title": "Deep Learning",
                    "language": "python",
                    "author": "Yoshua Bengio",
                    "price": 549,
                    "publish_time": "2016-11-18",
                    "description": "written by three experts in the field, deep learning is the only comprehensive book on the subject."
                
            ,
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "3",
                "_score": 1,
                "_source": 
                    "id": "3",
                    "title": "Core Java",
                    "language": "java",
                    "author": "Horstmann",
                    "price": 85.9,
                    "publish_time": "2016-06-01",
                    "description": "The book is aimed at experienced programmers who want to learn how to write useful Java applications and applets. "
                
            
        ]

范围查询

range query是范围查询，例如查询publish_time在"2016-01-01"到"2016-12-31"之间的文档：

GET englishbooks/_search


    "query":
        "range":
            "publish_time":
                "gte":"2016-01-01",
                "lte":"2016-12-31",
                "format":"yyyy-MM-dd"

篇幅所限，此处略去返回结果；

exists query

exists query返回的是字段中至少有一个非空值的文档：

GET englishbooks/_search


    "query":
        "exists":
            "field":"author"

前缀查询

用于查询某个字段是否以给定前缀开始：

GET englishbooks/_search


    "query":
        "prefix":
            "title":"cor"

以上请求可以查到title字段为"Core Java"的文档：


    "took": 6,
    "timed_out": false,
    "_shards": 
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    ,
    "hits": 
        "total": 1,
        "max_score": 1,
        "hits": [
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "3",
                "_score": 1,
                "_source": 
                    "id": "3",
                    "title": "Core Java",
                    "language": "java",
                    "author": "Horstmann",
                    "price": 85.9,
                    "publish_time": "2016-06-01",
                    "description": "The book is aimed at experienced programmers who want to learn how to write useful Java applications and applets. "
                
            
        ]

通配符查询

以下查询，可以搜到title字段中含有"core"的文档，另外需要注意的是，"?"匹配一个字符，"*"匹配零个或者多个字符：

GET englishbooks/_search


    "query":
        "wildcard":
            "title":"cor?"

正则表达式

使用属性regexp可以进行正则表达式查询，例如查找description字段带有4位数字的分词的文档：

GET englishbooks/_search


    "query":
        "regexp":
            "description":"[0-9]4"

查找结果如下，description字段中带有数字1986：


    "took": 4,
    "timed_out": false,
    "_shards": 
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    ,
    "hits": 
        "total": 1,
        "max_score": 1,
        "hits": [
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "2",
                "_score": 1,
                "_source": 
                    "id": "2",
                    "title": "Compilers",
                    "language": "c",
                    "author": "Alfred V.Aho",
                    "price": 62.5,
                    "publish_time": "2011-01-01",
                    "description": "In the time since the 1986 edition of this book, the world of compiler designhas changed significantly."
                
            
        ]

模糊查询(fuzzy query)

fuzzy是通过计算词项与文档的编辑距离来得到结果的，例如查找description字段还有分词"1986"的时候，不小心输入了"1987"，通过fuzzy查询也能得到结果，只是得分变低了，请求内容如下所示：

GET englishbooks/_search


    "query":
        "fuzzy":
            "description":"1987"

搜索到的文档如下所示，得分只有0.5942837，低于用"1986"查询的0.79237825：


    "took": 5,
    "timed_out": false,
    "_shards": 
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    ,
    "hits": 
        "total": 1,
        "max_score": 0.5942837,
        "hits": [
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "2",
                "_score": 0.5942837,
                "_source": 
                    "id": "2",
                    "title": "Compilers",
                    "language": "c",
                    "author": "Alfred V.Aho",
                    "price": 62.5,
                    "publish_time": "2011-01-01",
                    "description": "In the time since the 1986 edition of this book, the world of compiler designhas changed significantly."
                
            
        ]

需要注意的是，fuzzy查询时消耗资源较大；

复合查询

常用到的复合查询是bool query，可以用下表中的条件组合查询：

属性	作用
must	必须匹配，相当于SQL中的AND
should	可以匹配，相当于SQL中的OR
must_not	必须不匹配
filter	和must一样，但是不评分

以下条件，搜索的是title中带有java，但是不包含core的文档：

GET englishbooks/_search


    "query":
        "bool":
            "must":
                "term":"title":"java" 
            ,
            "must_not":[
            "term":"title":"core"
            ]

得到的文档中，带有core词项的已经被过滤了：


    "took": 3,
    "timed_out": false,
    "_shards": 
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    ,
    "hits": 
        "total": 1,
        "max_score": 0.5754429,
        "hits": [
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "4",
                "_score": 0.5754429,
                "_source": 
                    "id": "4",
                    "title": "Thinking in Java",
                    "language": "java",
                    "author": "Bruce Eckel",
                    "price": 70.1,
                    "publish_time": "2015-07-06",
                    "description": "Thinking in Java should be read cover to cover by every Java programmer, then kept close at hand for frequent reference. The exercises are challenging, and the chapter on Collections is superb!"
                
            
        ]

脚本查询

可用脚本进行查询，如下是查询价格大于100的所有文档：

GET englishbooks/_search


    "query":
        "bool":
            "must":
                "script":
                    "script":
                        "inline":"doc[price].value>500",
                        "lang":"painless"

得到的结果只有price大于500的文档：


    "took": 8,
    "timed_out": false,
    "_shards": 
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    ,
    "hits": 
        "total": 1,
        "max_score": 1,
        "hits": [
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "1",
                "_score": 1,
                "_source": 
                    "id": "1",
                    "title": "Deep Learning",
                    "language": "python",
                    "author": "Yoshua Bengio",
                    "price": 549,
                    "publish_time": "2016-11-18",
                    "description": "written by three experts in the field, deep learning is the only comprehensive book on the subject."
                
            
        ]

指定排序字段

默认的排序方式是按照评分来排序的（也就是相关度排序），可以用sort属性来设置排序字段，下面的请求指定了按照price字段降序排序：


    "query":
        "term":"title":"java"
    ,
    "sort":[
        "price":"order":"desc"
    ]

得到结果：


    "took": 4,
    "timed_out": false,
    "_shards": 
        "total": 5,
        "successful": 5,
        "skipped": 0,
        "failed": 0
    ,
    "hits": 
        "total": 2,
        "max_score": null,
        "hits": [
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "3",
                "_score": null,
                "_source": 
                    "id": "3",
                    "title": "Core Java",
                    "language": "java",
                    "author": "Horstmann",
                    "price": 85.9,
                    "publish_time": "2016-06-01",
                    "description": "The book is aimed at experienced programmers who want to learn how to write useful Java applications and applets. "
                ,
                "sort": [
                    85.9
                ]
            ,
            
                "_index": "englishbooks",
                "_type": "IT",
                "_id": "4",
                "_score": null,
                "_source": 
                    "id": "4",
                    "title": "Thinking in Java",
                    "language": "java",
                    "author": "Bruce Eckel",
                    "price": 70.1,
                    "publish_time": "2015-07-06",
                    "description": "Thinking in Java should be read cover to cover by every Java programmer, then kept close at hand for frequent reference. The exercises are challenging, and the chapter on Collections is superb!"
                ,
                "sort": [
                    70.1
                ]
            
        ]

以上就是常用的搜索操作了，至此，《elasticsearch实战三部曲》系列就全部完成，三篇文章列举的是一些常用的基本操作，希望能帮助读者您快速熟悉elasticsearch，后面咱们再一起深入实战；

欢迎关注51CTO博客：程序员欣宸

以上是关于elasticsearch实战三部曲之三：搜索操作的主要内容，如果未能解决你的问题，请参考以下文章

Java版人脸跟踪三部曲之三：编码实战

CoProcessFunction实战三部曲之三：定时器和侧输出

Go语言基准测试(benchmark)三部曲之三：提高篇

Docker下RabbitMQ四部曲之三：细说java开发

Docker下Prometheus和Grafana三部曲之三：自定义监控项开发和配置

案例实战Elasticsearch基本操作