ES基本CURD使用DSL搜索 高亮显示分页批量操作

Posted handler-刘

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了ES基本CURD使用DSL搜索 高亮显示分页批量操作相关的知识，希望对你有一定的参考价值。

目录

RESTful Api

        创建非结构化索引

        插入数据

        更新数据

        搜索数据

查询全部数据，响应（默认返回10条数据）

        DSL搜索

        高亮显示  highilght

        聚合

核心详解

        文档 

        查询响应

        批量操作

      _bulk操作

        映射

创建明确类型的索引

查看映射 

        结构化查询

                term查询

​​               terms查询

                range查询

range过滤 按照指定范围查询一批数据

                exists查询

​​

                bool查询​

         过滤查询​

RESTful Api

在ES中，提供了功能丰富的restful api 的操作，包括基本的CURD、创建索引、删除索引等操作。

        创建非结构化索引

在lucene中 ，创建索引是需要定义字段名称以及字段的类型的，在ES中提供了非结构化的索引，就是不需要创建索引结构，即可写入数据到索引中，实际上ES底层会进行结构化操作，此操作对用户是透明的。

#创建索引 http://127.0.0.1:9200/content

PUT /content

{
   "settings":{
       "index":{
           "number_of_shards":"2",  //#分片数
           "number_of_replicas":"0" //#副本数
       }
   }
}

#删除索引 http://127.0.0.1:9200/content
DELETE  /content

        插入数据

URL规则：
POST /索引/类型/{id}


POST    http://127.0.0.1:9200/content/user/100

{
  "id":100,
  "name":"张三",
  "age":18,
  "sex":"男"
}

说明：非结构化的索引，不需要时限创建，直接插入数据默认创建索引。

不指定ID插入数据 ， ES字段创建32位字符串长度id

URL规则：
POST /索引/类型/


POST    http://127.0.0.1:9200/content/user/

{
  "id":100,
  "name":"张三",
  "age":18,
  "sex":"男"
}

        更新数据

在ES中，文档数据是不为修改的，但是可以通过覆盖的方式进行更新。

PUT    http://127.0.0.1:9200/content/user/100

{
  "id":100,
  "name":"张三",
  "age":18,
  "sex":"未知"
}

                局部字段更新

#注意：这里多了 _update 标识
POST    http://127.0.0.1:9200/content/user/100/_update


#多一层 doc
{
    "doc":{
        "age":15
    }
}

        删除数据 （如果删除不存在的数据 状态码404）

_id

DELETE    http://127.0.0.1:9200/content/user/100

        搜索数据


GET    http://127.0.0.1:9200/content/user/100


#返回数据

{
    "_index": "content",
    "_type": "user",
    "_id": "100",
    "_version": 3,
    "found": true,
    "_source": {
        "id": 100,
        "name": "张三",
        "age": 15,
        "sex": "未知"
    }
}

查询全部数据，响应（默认返回10条数据）

_search

GET    http://127.0.0.1:9200/content/user/_search

返回数据：
{
    "took": 72,
    "timed_out": false,
    "_shards": {
        "total": 2,
        "successful": 2,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 1.0,
        "hits": [
            {
                "_index": "content",
                "_type": "user",
                "_id": "AXzQGuVL9AvrP5Vp8NAL",
                "_score": 1.0,
                "_source": {
                    "id": 101,
                    "name": "李三",
                    "age": 20,
                    "sex": "男"
                }
            },
            {
                "_index": "content",
                "_type": "user",
                "_id": "100",
                "_score": 1.0,
                "_source": {
                    "id": 100,
                    "name": "张三",
                    "age": 15,
                    "sex": "未知"
                }
            }
        ]
    }
}

 关键字查询数据

_search
# 查询年龄等于20

GET    http://127.0.0.1:9200/content/user/_search?q=age:20

返回数据：
{
    "took": 62,
    "timed_out": false,
    "_shards": {
        "total": 2,
        "successful": 2,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 1.0,
        "hits": [
            {
                "_index": "content",
                "_type": "user",
                "_id": "AXzQGuVL9AvrP5Vp8NAL",
                "_score": 1.0,
                "_source": {
                    "id": 101,
                    "name": "李三",
                    "age": 20,
                    "sex": "男"
                }
            }
        ]
    }
}

        DSL搜索

ES提供丰富且灵活的查询语音叫做DSL查询（QUERY DSL）允许你构建更加复杂、强大的查询，以Json请求形式出现

_search
# 查询年龄等于20

POST   http://127.0.0.1:9200/content/user/_search

请求数据：
{
    "query":{
        "match":{
            "age":20
        }
    }
}

返回数据：
{
    "took": 64,
    "timed_out": false,
    "_shards": {
        "total": 2,
        "successful": 2,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 1.0,
        "hits": [
            {
                "_index": "content",
                "_type": "user",
                "_id": "AXzQGuVL9AvrP5Vp8NAL",
                "_score": 1.0,
                "_source": {
                    "id": 101,
                    "name": "李三",
                    "age": 20,
                    "sex": "男"
                }
            }
        ]
    }
}
_search
# 查询条件：  age > 20 && sex ="男"

POST   http://127.0.0.1:9200/content/user/_search

请求数据：
{
    "query":{
        "bool":{
            "filter":{
                "range":{
                    "age":{
                        "gt":15
                    }
                }
            },
            "must":{
                "match":{
                    "sex":"男"
                }
            }

        }
    }
}

返回数据：
{
    "took": 26,
    "timed_out": false,
    "_shards": {
        "total": 2,
        "successful": 2,
        "failed": 0
    },
    "hits": {
        "total": 1,
        "max_score": 1.0,
        "hits": [
            {
                "_index": "content",
                "_type": "user",
                "_id": "AXzQGuVL9AvrP5Vp8NAL",
                "_score": 1.0,
                "_source": {
                    "id": 101,
                    "name": "李三",
                    "age": 20,
                    "sex": "男"
                }
            }
        ]
    }
}

_search
# 查询条件：  name  "李三 张三"

POST   http://127.0.0.1:9200/content/user/_search

请求数据：
{
    "query":{
         "match":{
            "name":"李三 张三"
         }
    }
}

返回数据：
{
    "took": 41,
    "timed_out": false,
    "_shards": {
        "total": 2,
        "successful": 2,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 0.48631853,
        "hits": [
            {
                "_index": "content",
                "_type": "user",
                "_id": "AXzQGuVL9AvrP5Vp8NAL",
                "_score": 0.48631853,
                "_source": {
                    "id": 101,
                    "name": "李三",
                    "age": 20,
                    "sex": "男"
                }
            },
            {
                "_index": "content",
                "_type": "user",
                "_id": "100",
                "_score": 0.48631853,
                "_source": {
                    "id": 100,
                    "name": "张三",
                    "age": 15,
                    "sex": "未知"
                }
            }
        ]
    }
}

        高亮显示  highilght

_search
# 查询条件：  name  "李三 张三"

POST   http://127.0.0.1:9200/content/user/_search

请求数据：
{
    "query":{
         "match":{
            "name":"李三 张三"
         }
    },
    "highlight":{
        "fields":{
            "name":{}
        }
    }
}

返回数据：
{
    "took": 46,
    "timed_out": false,
    "_shards": {
        "total": 2,
        "successful": 2,
        "failed": 0
    },
    "hits": {
        "total": 2,
        "max_score": 0.48631853,
        "hits": [
            {
                "_index": "content",
                "_type": "user",
                "_id": "AXzQGuVL9AvrP5Vp8NAL",
                "_score": 0.48631853,
                "_source": {
                    "id": 101,
                    "name": "李三",
                    "age": 20,
                    "sex": "男"
                },
                "highlight": {
                    "name": [
                        "<em>李</em><em>三</em>"
                    ]
                }
            },
            {
                "_index": "content",
                "_type": "user",
                "_id": "100",
                "_score": 0.48631853,
                "_source": {
                    "id": 100,
                    "name": "张三",
                    "age": 15,
                    "sex": "未知"
                },
                "highlight": {
                    "name": [
                        "<em>张</em><em>三</em>"
                    ]
                }
            }
        ]
    }
}

        聚合

在ES中，支持聚合操作，类似SQL中的 group by

_search
# 聚合： age

POST   http://127.0.0.1:9200/content/user/_search

请求数据：
{
    "aggs":{
        "all_interests":{
            "terms":{
                "field" :"age"
            }
        }
    }
}

返回数据：
{
    "took": 62,
    "timed_out": false,
    "_shards": {
        "total": 2,
        "successful": 2,
        "failed": 0
    },
    "hits": {
        "total": 5,
        "max_score": 1.0,
        "hits": [
            {
                "_index": "content",
                "_type": "user",
                "_id": "AXzQGuVL9AvrP5Vp8NAL",
                "_score": 1.0,
                "_source": {
                    "id": 101,
                    "name": "李三",
                    "age": 20,
                    "sex": "男"
                }
            },
            {
                "_index": "content",
                "_type": "user",
                "_id": "100",
                "_score": 1.0,
                "_source": {
                    "id": 100,
                    "name": "张三",
                    "age": 15,
                    "sex": "未知"
                }
            },
            {
                "_index": "content",
                "_type": "user",
                "_id": "AXzQPW829AvrP5Vp8NAN",
                "_score": 1.0,
                "_source": {
                    "id": 101,
                    "name": "李三",
                    "age": 20,
                    "sex": "男"
                }
            },
            {
                "_index": "content",
                "_type": "user",
                "_id": "AXzQPY279AvrP5Vp8NAO",
                "_score": 1.0,
                "_source": {
                    "id": 103,
                    "name": "李三",
                    "age": 20,
                    "sex": "男"
                }
            },
            {
                "_index": "content",
                "_type": "user",
                "_id": "AXzQPZ0T9AvrP5Vp8NAP",
                "_score": 1.0,
                "_source": {
                    "id": 104,
                    "name": "李三",
                    "age": 20,
                    "sex": "男"
                }
            }
        ]
    },
    "aggregations": {
        "all_interests": {
            "doc_count_error_upper_bound": 0,
            "sum_other_doc_count": 0,
            "buckets": [
                {
                    "key": 20,
                    "doc_count": 4
                },
                {
                    "key": 15,
                    "doc_count": 1
                }
            ]
        }
    }
}

核心详解

元数据（metadata）

一个文档不只有数据，还包括了元数据（metadata） 关于文档的信息，三个必须的元数据节点是：

节点说明
_index文档存储的地方
_type文档代表的对象的类
_id文档的唯一标识

        文档 

        查询响应

                pretty  （格式化输出）

? pretty 格式化输出数据

 http://127.0.0.1:9200/content/user/_search?pretty

                指定响应字段 （指定显示字段）

? _source=id,name 显示字段，输出数据

 http://127.0.0.1:9200/content/user/_search?_source=id,name

        判断是否存在

HEAD http://127.0.0.1:9200/content/user/100

#数据存在 状态码 返回 200

#数据不存在 状态码 返回 404

        批量操作

有些情况下可以通过批量操作减少网络请求，列如：批量查询、批量插入数据

                批量查询

_mget


POST   http://127.0.0.1:9200/content/user/_mget

请求数据：
{
    "ids":["100","103"]
}

返回数据：
{
    "docs": [
        {
            "_index": "content",
            "_type": "user",
            "_id": "100",
            "_version": 3,
            "found": true,
            "_source": {
                "id": 100,
                "name": "张三",
                "age": 15,
                "sex": "未知"
            }
        },
        {
            "_index": "content",
            "_type": "user",
            "_id": "103",
            "found": false
        }
    ]
}

                _bulk操作

 

批量插入
_bulk


POST   http://127.0.0.1:9200/content/user/_bulk

请求数据：
{"create":{"_index":"content","_type":"user","_id":1005}}
{"id":1005,"name":"name-015","age":24,"sex":"男"}
{"create":{"_index":"content","_type":"user","_id":1006}}
{"id":1006,"name":"name-016","age":18,"sex":"男"}
{"create":{"_index":"content","_type":"user","_id":1007}}
{"id":1007,"name":"name-017","age":36,"sex":"女""}
{"create":{"_index":"content","_type":"user","_id":1008}}
{"id":1008,"name":"name-018","age":65,"sex":"男"}

返回数据：
{
    "took": 398,
    "errors": true,
    "items": [
        {
            "create": {
                "_index": "content",
                "_type": "user",
                "_id": "1005",
                "_version": 1,
                "_shards": {
                    "total": 1,
                    "successful": 1,
                    "failed": 0
                },
                "status": 201
            }
        },
        {
            "create": {
                "_index": "content",
                "_type": "user",
                "_id": "1006",
                "_version": 1,
                "_shards": {
                    "total": 1,
                    "successful": 1,
                    "failed": 0
                },
                "status": 201
            }
        },
        {
            "create": {
                "_index": "content",
                "_type": "user",
                "_id": "1007",
                "status": 400,
                "error": {
                    "type": "mapper_parsing_exception",
                    "reason": "failed to parse",
                    "caused_by": {
                        "type": "json_parse_exception",
                        "reason": "Unexpected character ('\\"' (code 34)): was expecting comma to separate Object entries\\n at [Source: org.elasticsearch.common.io.stream.InputStreamStreamInput@218defa6; line: 1, column: 51]"
                    }
                }
            }
        },
        {
            "create": {
                "_index": "content",
                "_type": "user",
                "_id": "1008",
                "_version": 1,
                "_shards": {
                    "total": 1,
                    "successful": 1,
                    "failed": 0
                },
                "status": 201
            }
        }
    ]
}
批量删除
_bulk


POST   http://127.0.0.1:9200/content/user/_bulk

请求数据：
{"delete":{"_index":"content","_type":"user","_id":1005}}
{"delete":{"_index":"content","_type":"user","_id":1006}}
{"delete":{"_index":"content","_type":"user","_id":1007}}
{"delete":{"_index":"content","_type":"user","_id":1008}}


返回数据：
{
    "took": 115,
    "errors": false,
    "items": [
        {
            "delete": {
                "_index": "content",
                "_type": "user",
                "_id": "1005",
                "_version": 2,
                "_shards": {
                    "total": 1,
                    "successful": 1,
                    "failed": 0
                },
                "status": 200,
                "found": true
            }
        },
        {
            "delete": {
                "_index": "content",
                "_type": "user",
                "_id": "1006",
                "_version": 2,
                "_shards": {
                    "total": 1,
                    "successful": 1,
                    "failed": 0
                },
                "status": 200,
                "found": true
            }
        },
        {
            "delete": {
                "_index": "content",
                "_type": "user",
                "_id": "1007",
                "_version": 1,
                "_shards": {
                    "total": 1,
                    "successful": 1,
                    "failed": 0
                },
                "status": 404,
                "found": false
            }
        },
        {
            "delete": {
                "_index": "content",
                "_type": "user",
                "_id": "1008",
                "_version": 2,
                "_shards": {
                    "total": 1,
                    "successful": 1,
                    "failed": 0
                },
                "status": 200,
                "found": true
            }
        }
    ]
}

        分页

和SQL使用LIMIT关键字返回只有一页的结果一样。ES接收from和size参数：

size:结果数、默认10
from：跳过开始的结果数，默认0     （size-1）*from
GET /_search?size=5
GET /_search?size=5&from=5
GET /_search?size=5&from=10

        映射

创建明确类型的索引

#创建索引 http://127.0.0.1:9200/student

PUT /student

{
   "settings":{
       "index":{
           "number_of_shards":"2",  //#分片数
           "number_of_replicas":"0" //#副本数
       }
   },
   "mappings": {
        "user": {
            "properties": {
                "id": {
                    "type": "long"
                },
                "name": {
                    "type": "string"
                },
                "sex": {
                    "type": "string"
                },
                "mail": {
                    "type": "string"
                },
                "age": {
                    "type": "long"
                },
                "hobby":{
                    "type": "string"
                }
            }
        }   
    }
}

#删除索引 http://127.0.0.1:9200/student
DELETE  /student

查看映射 

GET  http://127.0.0.1:9200/content/user/_mapping

        结构化查询

                term查询

               terms查询

                range查询

range过滤 按照指定范围查询一批数据

                exists查询

                 match查询

                bool查询

         过滤查询

 

 

以上是关于ES基本CURD使用DSL搜索 高亮显示分页批量操作的主要内容，如果未能解决你的问题，请参考以下文章

商城项目17_es的DSL特定语言matchbooltermtermsaggsfromsizerangesort排序查询高亮显示

商城项目17_es的DSL特定语言matchbooltermtermsaggsfromsizerangesort排序查询高亮显示

ElasticSearch_全文搜索引擎目录

ElasticSearch_全文搜索引擎目录

ElasticSearch仿京东搜索实战

SpringBoot+ElasticSearch 实现模糊查询，批量CRUD，排序，分页，高亮