elk快速入门-在kibana中如何使用devtools操作elasticsearch

Posted 2020-11-15 lonelyxmas

原文:elk快速入门-在kibana中如何使用devtools操作elasticsearch

在kibana中如何使用devtools操作elasticsearch：
前言：
首先需要安装elasticsearch,kibana ,下载地址 https://www.elastic.co/cn/downloads/
权威指南：https://www.elastic.co/guide/cn/index.html
视频：https://www.elastic.co/cn/webinars/getting-started-elasticsearch?elektra=home&storm=sub1
https://www.elastic.co/cn/webinars/getting-started-kibana?elektra=home&storm=sub2
https://www.elastic.co/cn/webinars/getting-started-logstash
1.登录到kibana:http://localhost:5601/app/kibana#/dev_tools/console?_g=()
2.打开devtools
3.基本使用：
获取es基本信息，效果与直接访问http://localhost:9200/一样， 在devtools中可以省去http://localhost:9200这一截
GET /

结果==>>

{
"name" : "DESKTOP-1HUG1AS",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "VLtxooalQyKdSzQp0V_gcg",
"version" : {
"number" : "7.1.0",
"build_flavor" : "default",
"build_type" : "zip",
"build_hash" : "606a173",
"build_date" : "2019-05-16T00:43:15.323135Z",
"build_snapshot" : false,
"lucene_version" : "8.0.0",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}

 

GET /_cat/health

结果==>>

=^.^=
/_cat/allocation
/_cat/shards
/_cat/shards/{index}
/_cat/master
/_cat/nodes
/_cat/tasks
/_cat/indices
/_cat/indices/{index}
/_cat/segments
/_cat/segments/{index}
/_cat/count
/_cat/count/{index}
/_cat/recovery
/_cat/recovery/{index}
/_cat/health
/_cat/pending_tasks
/_cat/aliases
/_cat/aliases/{alias}
/_cat/thread_pool
/_cat/thread_pool/{thread_pools}
/_cat/plugins
/_cat/fielddata
/_cat/fielddata/{fields}
/_cat/nodeattrs
/_cat/repositories
/_cat/snapshots/{repository}
/_cat/templates

新增自己的数据：（如果使用POST test001/doc不带1,系统会每次自己生产一个_id）
POST test001/doc/1
{
"user":"zhangsan",
"age":18,
"city":"深圳"
}
查询
GET test001/doc/1

更新
put test001/doc/1
{
"user":"zhangsan",
"age":18,
"city":"sz",
"location":{
"jd":12,
"wd":34
}
}

删除单个
DELETE test001/doc/1

删除所有
DELETE test001

检索所有数据
GET test001/_search

批量新增： 第一行表示操作，第二行表示数据内容，注意数据内容需要在一行，不能跨行，否则会新增不成功
POST _bulk
{"index":{"_index":"test002","_type":"doc"}}
{"user":"zhangsan", "age":30,"message":"happy birthday","city":"北京","location":{"jd":12,"wd":34}}
{"index":{"_index":"test002","_type":"doc"}}
{"user":"lisi", "age":30,"message":"happy birthday","city":"上海","location":{"jd":12,"wd":34}}
{"index":{"_index":"test002","_type":"doc"}}
{"user":"wangwu", "age":35,"message":"Happy birthday","city":"深圳","location":{"jd":12,"wd":34}}
{"index":{"_index":"test002","_type":"doc"}}
{"user":"zhaoliu", "age":40,"message":"birthday happy","city":"深圳","location":{"jd":12,"wd":34}}

根据单个条件查询，city为"深圳"的数据
GET test002/_search
{
"query": {"match": {
"city": "深圳"
}}
}

根据多个条件查询，city为"深圳" 并且age=35的数据
GET test002/_search
{
"query": {
"bool": {"must": [
{"match": {
"city": "深圳"
}},{"match": {
"age": "35"
}}
]}
}
}

根据单个条件查询（取反操作），city不为"深圳"的数据
GET test002/_search
{
"query": {"bool": {"must_not": [
{"match": {
"city": "深圳"
}}
]}}
}

查询或的条件，city为"上海"或city为"深圳"的数据
GET test002/_search
{
"query": {"bool": {"should": [
{"match": {
"city": "上海"
}},{"match": {
"city": "深圳"
}}
]}}
}

如果只想查询数量，不想查询数据，只需要将_search换成_count即可
GET test002/_count 不带条件
或者
GET test002/_count
{
"query": {"bool": {"should": [
{"match": {
"city": "上海"
}},{"match": {
"city": "深圳"
}}
]}}
}

范围查询range,查询age为30到35岁的记录
GET test002/_search
{
"query": {
"range": {
"age": {
"gte": 30,
"lte": 35
}
}
}
}

排序sort,对age降序排序
GET test002/_search
{
"query": {
"range": {
"age": {
"gte": 30,
"lte": 35
}
}
},"sort": [
{
"age": {
"order": "desc"
}
}
]
}

对某个字段如message查询关键字包含happy birthday的数据,会查询出birthday happy的数据
GET test002/_search
{
"query": {
"match": {
"message": "happy birthday"
}
}
}

而使用match_phrase，就不会查询birthday happy的数据了
GET test002/_search
{
"query": {
"match_phrase": {
"message": "happy birthday"
}
}
}

对关键字高亮highlight,如对message进行高亮。 es会加上em的标签如："<em>happy</em> <em>birthday</em>"
GET test002/_search
{
"query": {
"match_phrase": {
"message": "happy birthday"
}
},
"highlight": {
"fields": {
"message":{}
}
}
}

对查询结果聚合使用aggs，如想统计20-30，30-40,40-100岁的人分别有多少个 。查看aggregations结果
GET test002/_search
{
"aggs": {
"age": {
"range": {
"field": "age",
"ranges": [
{
"from": 20,
"to": 30
},
{
"from": 30,
"to": 40
},
{
"from": 40,
"to": 100
}
]
}
}
}
}

如果不想看到详情数据，可以增加一个属性"size":0 ，在hits中就看不到数据了
GET test002/_search
{
"size": 0,
"aggs": {
"age": {
"range": {
"field": "age",
"ranges": [
{
"from": 20,
"to": 30
},
{
"from": 30,
"to": 40
},
{
"from": 40,
"to": 100
}
]
}
}
}
}

统计某个字段个数，使用aggs和terms，类似group by分组
GET test002/_search
{
"size": 0,
"aggs": {
"city": {
"terms": {
"field": "city.keyword",
"size": 10
}
}
}
}

 

type:text的字段默认会有analyzer:standard的属性（内置分析器）

查看Happy Birthday会被分析器如何分析
GET test002/_analyze
{
"text": ["Happy Birthday"],
"analyzer": "standard"
}

结果==>>
{
"tokens" : [
{
"token" : "happy",
"start_offset" : 0,
"end_offset" : 5,
"type" : "<ALPHANUM>",
"position" : 0
},
{
"token" : "birthday",
"start_offset" : 6,
"end_offset" : 14,
"type" : "<ALPHANUM>",
"position" : 1
}
]
}
可以看到Happy Birthday 被拆分成happy 和 birthday 并且都转成小写了

如果之间带了. 那么是不会做拆分的，只会转成小鞋
GET test002/_analyze
{
"text": ["Happy.Birthday"]
}
结果==>>
{
"tokens" : [
{
"token" : "happy.birthday",
"start_offset" : 0,
"end_offset" : 14,
"type" : "<ALPHANUM>",
"position" : 0
}
]
}

如果之间带了. 还有什么办法可以拆分吗？使用simple分析器"analyzer": "simple"
GET test002/_analyze
{
"text": ["Happy.Birthday"],
"analyzer": "simple"
}
结果==>>
{
"tokens" : [
{
"token" : "happy",
"start_offset" : 0,
"end_offset" : 5,
"type" : "word",
"position" : 0
},
{
"token" : "birthday",
"start_offset" : 6,
"end_offset" : 14,
"type" : "word",
"position" : 1
}
]
}

tokenizer和analyzer类似。"tokenizer": "standard"会做拆分，而"tokenizer": "keyword"会当做一个整体
GET test002/_analyze
{
"text": ["Happy Birthday"],
"tokenizer": "standard"
}
结果==>>
{
"tokens" : [
{
"token" : "Happy",
"start_offset" : 0,
"end_offset" : 5,
"type" : "<ALPHANUM>",
"position" : 0
},
{
"token" : "Birthday",
"start_offset" : 6,
"end_offset" : 14,
"type" : "<ALPHANUM>",
"position" : 1
}
]
}

GET test002/_analyze
{
"text": ["Happy Birthday"],
"tokenizer": "keyword"
}

结果==>>
{
"tokens" : [
{
"token" : "Happy Birthday",
"start_offset" : 0,
"end_offset" : 14,
"type" : "word",
"position" : 0
}
]
}

可以看到上面的结果没有转成小写，如果要转成小写，增加"filter": ["lowercase"]
GET test002/_analyze
{
"text": ["Happy Birthday"],
"tokenizer": "standard",
"filter": ["lowercase"]
}
结果==>>
{
"tokens" : [
{
"token" : "happy",
"start_offset" : 0,
"end_offset" : 5,
"type" : "<ALPHANUM>",
"position" : 0
},
{
"token" : "birthday",
"start_offset" : 6,
"end_offset" : 14,
"type" : "<ALPHANUM>",
"position" : 1
}
]
}

 

查询数据类型
GET test002/_mapping
"type"为keyword代表不可拆分不能做分词是一个整体，text代表可以做分词

设置分片数
PUT test003
{
"settings": {"number_of_shards": 1}
}

设置_mapping 地理位置location字段为geo_point
PUT test003/_mapping
{
"properties": {
"user":{
"type": "text",
"fields":{
"keyword":{
"type":"keyword",
"ignore_above":256
}
}
},
"city":{
"type": "keyword"
},
"location":{
"type": "geo_point"
},
"message":{
"type": "text"
}

}
}

新增数据
POST _bulk
{"index":{"_index":"test003","_type":"doc"}}
{"user":"zhangsan", "age":30,"message":"happy birthday","city":"北京","location":{"lat":30,"lon":40}}
{"index":{"_index":"test003","_type":"doc"}}
{"user":"lisi", "age":30,"message":"happy birthday","city":"上海","location":{"lat":38.970718,"lon":116.325747}}
{"index":{"_index":"test003","_type":"doc"}}
{"user":"wangwu", "age":35,"message":"Happy birthday","city":"深圳","location":{"lat":37.970718,"lon":116.325747}}
{"index":{"_index":"test003","_type":"doc"}}
{"user":"zhaoliu", "age":40,"message":"birthday happy","city":"深圳","location":{"lat":36.970718,"lon":116.325747}}