Elasticsearch初探
Posted yoyotl
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Elasticsearch初探相关的知识，希望对你有一定的参考价值。
一、安装（Linux篇）
现在的开源软件越来越成熟，都打好包了，下载后直接解压就可以测试了。
1 curl -L -O https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.3.1.tar.gz 2 tar -zxvf elasticsearch-6.3.1.tar.gz 3 cd elasticsearch-6.3.1/bin/ 4 ./elasticsearch
注意：elasticsearch不允许使用root权限启动，所以使用root下载解压的同学，要chown改变一下文件夹的用户权限。
看到如下信息就说明启动成功了。
[2018-07-12T11:07:00,877][INFO ][o.e.n.Node ] [] initializing ... [2018-07-12T11:07:00,922][INFO ][o.e.e.NodeEnvironment ] [4LdtI3q] using [1] data paths, mounts [[/home (/dev/mapper/cl-home)]], net usable_space [229.9gb], net total_space [240.9gb], types [xfs] [2018-07-12T11:07:00,923][INFO ][o.e.e.NodeEnvironment ] [4LdtI3q] heap size [990.7mb], compressed ordinary object pointers [true] [2018-07-12T11:07:00,938][INFO ][o.e.n.Node ] [4LdtI3q] node name derived from node ID [4LdtI3q-T2ehl0Z6Xv6cOA]; set [node.name] to override [2018-07-12T11:07:00,939][INFO ][o.e.n.Node ] [4LdtI3q] version[6.3.1], pid[21521], build[default/tar/eb782d0/2018-06-29T21:59:26.107521Z], OS[Linux/3.10.0-514.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_112/25.112-b15] [2018-07-12T11:07:00,939][INFO ][o.e.n.Node ] [4LdtI3q] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.af3cuwkJ, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=data, -XX:ErrorFile=logs/hs_err_pid%p.log, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:logs/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Des.path.home=/home/lings/elastic/elasticsearch-6.3.1, -Des.path.conf=/home/lings/elastic/elasticsearch-6.3.1/config, -Des.distribution.flavor=default, -Des.distribution.type=tar] [2018-07-12T11:07:02,794][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [aggs-matrix-stats] [2018-07-12T11:07:02,794][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [analysis-common] [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [ingest-common] [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [lang-expression] [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [lang-mustache] [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [lang-painless] [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [mapper-extras] [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [parent-join] [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [percolator] [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [rank-eval] [2018-07-12T11:07:02,795][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [reindex] [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [repository-url] [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [transport-netty4] [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [tribe] [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-core] [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-deprecation] [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-graph] [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-logstash] [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-ml] [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-monitoring] [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-rollup] [2018-07-12T11:07:02,796][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-security] [2018-07-12T11:07:02,797][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-sql] [2018-07-12T11:07:02,797][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-upgrade] [2018-07-12T11:07:02,797][INFO ][o.e.p.PluginsService ] [4LdtI3q] loaded module [x-pack-watcher] [2018-07-12T11:07:02,797][INFO ][o.e.p.PluginsService ] [4LdtI3q] no plugins loaded [2018-07-12T11:07:05,500][INFO ][o.e.x.s.a.s.FileRolesStore] [4LdtI3q] parsed [0] roles from file [/home/lings/elastic/elasticsearch-6.3.1/config/roles.yml] [2018-07-12T11:07:05,896][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/21588] [[email protected]109] controller (64 bit): Version 6.3.1 (Build 4d0b8f0a0ef401) Copyright (c) 2018 Elasticsearch BV [2018-07-12T11:07:06,171][DEBUG][o.e.a.ActionModule ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security [2018-07-12T11:07:06,425][INFO ][o.e.d.DiscoveryModule ] [4LdtI3q] using discovery type [zen] [2018-07-12T11:07:07,061][INFO ][o.e.n.Node ] [4LdtI3q] initialized [2018-07-12T11:07:07,061][INFO ][o.e.n.Node ] [4LdtI3q] starting ... [2018-07-12T11:07:07,194][INFO ][o.e.t.TransportService ] [4LdtI3q] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300} [2018-07-12T11:07:07,216][WARN ][o.e.b.BootstrapChecks ] [4LdtI3q] max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] [2018-07-12T11:07:07,216][WARN ][o.e.b.BootstrapChecks ] [4LdtI3q] max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] [2018-07-12T11:07:10,259][INFO ][o.e.c.s.MasterService ] [4LdtI3q] zen-disco-elected-as-master ([0] nodes joined)[, ], reason: new_master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=16658751488, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} [2018-07-12T11:07:10,264][INFO ][o.e.c.s.ClusterApplierService] [4LdtI3q] new_master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=16658751488, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}, reason: apply cluster state (from master [master {4LdtI3q}{4LdtI3q-T2ehl0Z6Xv6cOA}{y7E-EJNzQKq7Xj5-B-okOA}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=16658751488, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true} committed version [1] source [zen-disco-elected-as-master ([0] nodes joined)[, ]]]) [2018-07-12T11:07:10,279][INFO ][o.e.x.s.t.n.SecurityNetty4HttpServerTransport] [4LdtI3q] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200} [2018-07-12T11:07:10,280][INFO ][o.e.n.Node ] [4LdtI3q] started [2018-07-12T11:07:10,622][WARN ][o.e.x.s.a.s.m.NativeRoleMappingStore] [4LdtI3q] Failed to clear cache for realms [[]] [2018-07-12T11:07:10,636][INFO ][o.e.l.LicenseService ] [4LdtI3q] license [d717e767-fe3f-4b36-bc85-cad6dbde3efc] mode [basic] - valid [2018-07-12T11:07:10,644][INFO ][o.e.g.GatewayService ] [4LdtI3q] recovered [2] indices into cluster_state [2018-07-12T11:07:10,956][INFO ][o.e.c.r.a.AllocationService] [4LdtI3q] Cluster health status changed from [RED] to [YELLOW] (reason: [shards started [[customer][2], [customer][3]] ...]).
二、REST使用示例
1. 查看集群的状态 curl -X GET "localhost:9200/_cat/health?v" epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent 1531365002 11:10:02 elasticsearch yellow 1 1 10 10 0 0 10 0 - 50.0% 注意这个status的状态是yellow，理想的状态应该是green，原因是没有配置集群。 2. 查看节点的状态 curl -X GET "localhost:9200/_cat/nodes?v" ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name 127.0.0.1 20 57 3 0.07 0.06 0.06 mdi * 4LdtI3q 3. 查看所有索引（类比mysql的库）的状态 curl -X GET ‘http://localhost:9200/_cat/indices?v‘ health status index uuid pri rep docs.count docs.deleted store.size pri.store.size yellow open bank BO1IYzoXSpGGyYP3XKo4bA 5 1 1000 0 475.1kb 475.1kb yellow open customer -8lF3GAFT1WI43gR9DiYpw 5 1 5 0 17.7kb 17.7kb 有两个索引bank和customer，一个1000条数据，一个有5条数据。 4. 查看索引（customer）的状态 curl -X PUT "localhost:9200/customer?pretty" { "error" : { "root_cause" : [ { "type" : "resource_already_exists_exception", "reason" : "index [customer/-8lF3GAFT1WI43gR9DiYpw] already exists", "index_uuid" : "-8lF3GAFT1WI43gR9DiYpw", "index" : "customer" } ], "type" : "resource_already_exists_exception", "reason" : "index [customer/-8lF3GAFT1WI43gR9DiYpw] already exists", "index_uuid" : "-8lF3GAFT1WI43gR9DiYpw", "index" : "customer" }, "status" : 400 } 5. 更新（不存在时则增加）索引的document（类比Mysql的行） curl -X PUT "localhost:9200/customer/_doc/1?pretty" -H ‘Content-Type: application/json‘ -d‘ { "name": "John Doe" } ‘ 响应如下： { "_index" : "customer", "_type" : "_doc", "_id" : "1", "_version" : 8, //每更新一次版本记录就+1 "result" : "updated", "_shards" : { "total" : 2, "successful" : 1, "failed" : 0 }, "_seq_no" : 7, "_primary_term" : 4 } 6. 增加索引的document（和PUT增加的区别就是不用指定document的id，会随机生成一个） curl -X POST "localhost:9200/customer/_doc?pretty" -H ‘Content-Type: application/json‘ -d‘ { "name": "Jane Doe" } ‘ 响应如下： { "_index" : "customer", "_type" : "_doc", "_id" : "Bu1tjWQBZa8_pSJjP5AF", //随机生成的id "_version" : 1, "result" : "created", "_shards" : { "total" : 2, "successful" : 1, "failed" : 0 }, "_seq_no" : 8, "_primary_term" : 4 } 7. 查看指定的document curl -X GET "localhost:9200/customer/_doc/1?pretty" 响应结果 { "_index" : "customer", "_type" : "_doc", "_id" : "1", "_version" : 8, "found" : true, "_source" : { "name" : "John Doe" } } 8. 删除指定的document curl -X DELETE "localhost:9200/customer/_doc/B-1ujWQBZa8_pSJjH5CC?pretty" 响应结果： { "_index" : "customer", "_type" : "_doc", "_id" : "B-1ujWQBZa8_pSJjH5CC", "_version" : 3, "result" : "not_found", "_shards" : { "total" : 2, "successful" : 1, "failed" : 0 }, "_seq_no" : 7, "_primary_term" : 4 } 9. 批量插入文本数据 curl -H "Content-Type: application/json" -XPOST "localhost:9200/bank/_doc/_bulk?pretty&refresh" --data-binary "@/home/lings/accounts.json" /home/lings/accounts.json为json文件的存放路径 10. 按照account_number升幂排序 curl -X GET "localhost:9200/bank/_search?q=*&sort=account_number:asc&pretty" 完全等效于这个命令 curl -X GET "localhost:9200/bank/_search" -H ‘Content-Type: application/json‘ -d‘ { "query": { "match_all": {} }, "sort": [ { "account_number": "asc" } ] } ‘ 响应结果： { "took" : 3, //查询花费了3ms "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 1000, //一共有1000条数据 "max_score" : null, "hits" : [ { "_index" : "bank", "_type" : "_doc", "_id" : "0", "_score" : null, "_source" : { "account_number" : 0, "balance" : 16623, "firstname" : "Bradshaw", "lastname" : "Mckenzie", "age" : 29, "gender" : "F", "address" : "244 Columbus Place", "employer" : "Euron", "email" : "[email protected]", "city" : "Hobucken", "state" : "CO" }, "sort" : [ 0 ] }, ......省略中间数据 { "_index" : "bank", "_type" : "_doc", "_id" : "9", "_score" : null, "_source" : { "account_number" : 9, "balance" : 24776, "firstname" : "Opal", "lastname" : "Meadows", "age" : 39, "gender" : "M", "address" : "963 Neptune Avenue", "employer" : "Cedward", "email" : "[email protected]", "city" : "Olney", "state" : "OH" }, "sort" : [ 9 //一共只返回了10个document ] } ] } } 11. 查询住址包含Street单词的所有document curl -X GET "localhost:9200/bank/_search?pretty" -H ‘Content-Type: application/json‘ -d‘ { "query": { "match": {"address":"Street"} }, "sort": [ { "account_number": "desc" } ] } ‘ 响应如下： { "took" : 9, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 385, //满足条件的385个，总共还是返回10个document "max_score" : null, "hits" : [ { "_index" : "bank", "_type" : "_doc", "_id" : "996", "_score" : null, "_source" : { "account_number" : 996, "balance" : 17541, "firstname" : "Andrews", "lastname" : "Herrera", "age" : 30, ...... 12. 查询住址包含Street单词的降幂排序后的2个document curl -X GET "localhost:9200/bank/_search?pretty" -H ‘Content-Type: application/json‘ -d‘ { "query": { "match": {"address":"Street"} }, "sort": [ { "account_number": "desc" } ], "size":2 } ‘ 响应如下： { "took" : 2, "timed_out" : false, "_shards" : { "total" : 5, "successful" : 5, "skipped" : 0, "failed" : 0 }, "hits" : { "total" : 385, "max_score" : null, "hits" : [ { "_index" : "bank", "_type" : "_doc", "_id" : "996", "_score" : null, "_source" : { "account_number" : 996, "balance" : 17541, "firstname" : "Andrews", "lastname" : "Herrera", "age" : 30, "gender" : "F", "address" : "570 Vandam Street", "employer" : "Klugger", "email" : "[email protected]", "city" : "Whitehaven", "state" : "MN" }, "sort" : [ 996 ] }, { "_index" : "bank", "_type" : "_doc", "_id" : "994", "_score" : null, "_source" : { "account_number" : 994, "balance" : 33298, "firstname" : "Madge", "lastname" : "Holcomb", "age" : 31, "gender" : "M", "address" : "612 Hawthorne Street", "employer" : "Escenta", "email" : "[email protected]", "city" : "Alafaya", "state" : "OR" }, "sort" : [ 994 ] } ] } }
以上是关于Elasticsearch初探的主要内容，如果未能解决你的问题，请参考以下文章