Elasticsearch-5.5.0以及elasticsearch-analysis-ik安装使用

Posted iteacat

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Elasticsearch-5.5.0以及elasticsearch-analysis-ik安装使用相关的知识,希望对你有一定的参考价值。

  最近项目中要求对文章进行全文检索,所以选择了elasticsearch作为全文检索引擎。在安装以及使用的过程中也是碰到了很多的坑,从最开始的7.3.2换到6.4.3再换到5.5.0,最后终于可以

使用了,太不容易了,所以开篇帖子记录下,希望能帮助到刚入门es的朋友们。

  下面介绍如何在Centos7上安装es,以及中文分词工具ik。因Elasticsearch是基于Java写的,所以它的运行环境中需要java的支持,在Linux下执行命令:java -version,检查JDK是否安装。如果没有安装,请安装JDK1.8以上版本。

一、es安装

1.下载elasticsearch压缩包

可前往 https://learnku.com/articles/33403 这个网址下载。选择5.5.0版本elasticsearch-5.5.0.zip即可。

2.解压elasticsearch压缩包

将压缩包拷贝之/usr/local/ 目录下,执行解压名称。

cp elasticsearch-5.5.0.zip /usr/local/
cd /usr/local
unzip elasticsearch-5.5.0.zip 

3.修改elasticsearch.yml配置文件

因为Elasticsearch5.0之后,不能使用root账户启动,所以我们要创建专门的elasticsearch用户组并授予文件夹权限。

vi /usr/local/elasticsearch-5.5.0/config/elasticsearch.yml
//插入如下内容#集群的名称
cluster.name: knowgram
# 节点的名称
node.name: knowgram_node1
#配置你存数据的目录
path.data: /usr/local/elasticsearch-5.5.0/data
# 配置你存日志的目录
path.logs: /usr/local/elasticsearch-5.5.0/logs
# Lock the memory on startup: 这两个配置用于解决后面报错问题,否则会报错
bootstrap.memory_lock: false
bootstrap.system_call_filter: false
# Set the bind address to a specific IP (IPv4 or IPv6):
#需要配置成本面的ip,否则外部无法访问
network.host: 0.0.0.0
transport.tcp.port: 9300
#设置跨域以及端口
http.port: 9200
http.cors.enabled: true
http.cors.allow-origin: "*"

4.创建一个elasticsearch组和账户

因为Elasticsearch5.0之后,不能使用root账户启动,所以我们要创建专门的elasticsearch用户组并授予文件夹权限。

groupadd elasticsearch
useradd  elasticsearch -g elasticsearch -p elasticsearch
chown -R elasticsearch:elasticsearch /usr/local/elasticsearch-5.5.0

5.切换到elasticsearch用户并启动elasticsearch

cd /usr/local/elasticsearch5.5.0/
su elasticsearch
./bin/elasticsearch

//或用以下命令
su - elasticsearch -c "/usr/local/elasticsearch-5.5.0/bin/elasticsearch -d"

 若看到输出started表示启动成功,若没有启动成功参考后面常见问题

[2019-11-05T12:34:09,961][INFO ][o.e.n.Node               ] [node_1] initializing ...
[2019-11-05T12:34:10,113][INFO ][o.e.e.NodeEnvironment    ] [node_1] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [46gb], net total_space [59.9gb], spins? [unknown], types [rootfs]
[2019-11-05T12:34:10,116][INFO ][o.e.e.NodeEnvironment    ] [node_1] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-11-05T12:34:10,125][INFO ][o.e.n.Node               ] [node_1] node name [node_1], node ID [b7PKnpL8TBqfX_j6N0ydyQ]
[2019-11-05T12:34:10,125][INFO ][o.e.n.Node               ] [node_1] version[5.5.0], pid[17571], build[260387d/2017-06-30T23:16:05.735Z], OS[Linux/3.10.0-957.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_201/25.201-b09]
[2019-11-05T12:34:10,125][INFO ][o.e.n.Node               ] [node_1] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+DisableExplicitGC, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/local/elasticsearch-5.5.0]
[2019-11-05T12:34:14,197][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [aggs-matrix-stats]
[2019-11-05T12:34:14,198][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [ingest-common]
[2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [lang-expression]
[2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [lang-groovy]
[2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [lang-mustache]
[2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [lang-painless]
[2019-11-05T12:34:14,201][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [parent-join]
[2019-11-05T12:34:14,201][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [percolator]
[2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [reindex]
[2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [transport-netty3]
[2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [transport-netty4]
[2019-11-05T12:34:14,203][INFO ][o.e.p.PluginsService     ] [node_1] loaded plugin [analysis-ik]
[2019-11-05T12:34:21,588][INFO ][o.e.d.DiscoveryModule    ] [node_1] using discovery type [zen]
[2019-11-05T12:34:24,056][INFO ][o.e.n.Node               ] [node_1] initialized
[2019-11-05T12:34:24,057][INFO ][o.e.n.Node               ] [node_1] starting ...
[2019-11-05T12:34:24,930][INFO ][o.e.t.TransportService   ] [node_1] publish_address {192.168.16.152:9300}, bound_addresses {[::]:9300}
[2019-11-05T12:34:25,035][INFO ][o.e.b.BootstrapChecks    ] [node_1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2019-11-05T12:34:28,268][INFO ][o.e.c.s.ClusterService   ] [node_1] new_master {node_1}{b7PKnpL8TBqfX_j6N0ydyQ}{8rQez1gDSturfNgSHluIyw}{192.168.16.152}{192.168.16.152:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2019-11-05T12:34:28,651][INFO ][o.e.h.n.Netty4HttpServerTransport] [node_1] publish_address {192.168.16.152:9200}, bound_addresses {[::]:9200}
[2019-11-05T12:34:28,651][INFO ][o.e.n.Node               ] [node_1] started //看到这条信息表示启动成功
[2019-11-05T12:34:28,676][INFO ][o.e.g.GatewayService     ] [node_1] recovered [0] indices into cluster_state

6.常见问题处理
问题一

max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]

max number of threads [1024] for user [es] likely too low, increase to at least [2048]


max file descriptors [4096] for elasticsearch process likely too low, increase to at least [65536]
max number of threads [1024] for user [es] likely too low, increase to at least [2048]

解决方式:更换到root用户,编辑limits.conf 配置文件

vi /etc/security/limits.conf

//添加如下内容:
* soft nofile 65536
* hard nofile 131072
* soft nproc 2048
* hard nproc 4096

  

问题二

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]  

max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]  

解决方式:修改 /etc/sysctl.conf 配置文件,增加如下配置

vi /etc/sysctl.conf
//追加如下内容
vm.max_map_count=655360

 

二、安装elasticsearch-analysis-ik

1.下载对应的版本IK

前往github下载对应elasticsearch对应的ik版本,我安装的是elasticsearch5.5.0,所以选择对应5.5.0版本

2.elasticsearch的plugins文件夹下新建ik目录

mkdir /usr/local/elasticsearch-5.5.0/plugins/ik

3.将压缩包复制到ik目录下并解压

cp elasticsearch-analysis-ik-5.5.0.zip /usr/local/elasticsearch-5.5.0/plugins/ik
cd /usr/local/elasticsearch-5.5.0/plugins/ik
unzip elasticsearch-analysis-ik-5.5.0.zip

最后重启elasticsearch,看到loaded plugin [analysis-ik]标识插件加载完成。

[2019-11-05T12:34:09,961][INFO ][o.e.n.Node               ] [node_1] initializing ...
[2019-11-05T12:34:10,113][INFO ][o.e.e.NodeEnvironment    ] [node_1] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [46gb], net total_space [59.9gb], spins? [unknown], types [rootfs]
[2019-11-05T12:34:10,116][INFO ][o.e.e.NodeEnvironment    ] [node_1] heap size [1.9gb], compressed ordinary object pointers [true]
[2019-11-05T12:34:10,125][INFO ][o.e.n.Node               ] [node_1] node name [node_1], node ID [b7PKnpL8TBqfX_j6N0ydyQ]
[2019-11-05T12:34:10,125][INFO ][o.e.n.Node               ] [node_1] version[5.5.0], pid[17571], build[260387d/2017-06-30T23:16:05.735Z], OS[Linux/3.10.0-957.el7.x86_64/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_201/25.201-b09]
[2019-11-05T12:34:10,125][INFO ][o.e.n.Node               ] [node_1] JVM arguments [-Xms2g, -Xmx2g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+DisableExplicitGC, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -Djdk.io.permissionsUseCanonicalPath=true, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j.skipJansi=true, -XX:+HeapDumpOnOutOfMemoryError, -Des.path.home=/usr/local/elasticsearch-5.5.0]
[2019-11-05T12:34:14,197][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [aggs-matrix-stats]
[2019-11-05T12:34:14,198][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [ingest-common]
[2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [lang-expression]
[2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [lang-groovy]
[2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [lang-mustache]
[2019-11-05T12:34:14,199][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [lang-painless]
[2019-11-05T12:34:14,201][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [parent-join]
[2019-11-05T12:34:14,201][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [percolator]
[2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [reindex]
[2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [transport-netty3]
[2019-11-05T12:34:14,202][INFO ][o.e.p.PluginsService     ] [node_1] loaded module [transport-netty4]
[2019-11-05T12:34:14,203][INFO ][o.e.p.PluginsService     ] [node_1] loaded plugin [analysis-ik] //看到这个标识插件加载成功
[2019-11-05T12:34:21,588][INFO ][o.e.d.DiscoveryModule    ] [node_1] using discovery type [zen]
[2019-11-05T12:34:24,056][INFO ][o.e.n.Node               ] [node_1] initialized
[2019-11-05T12:34:24,057][INFO ][o.e.n.Node               ] [node_1] starting ...
[2019-11-05T12:34:24,930][INFO ][o.e.t.TransportService   ] [node_1] publish_address {192.168.16.152:9300}, bound_addresses {[::]:9300}
[2019-11-05T12:34:25,035][INFO ][o.e.b.BootstrapChecks    ] [node_1] bound or publishing to a non-loopback or non-link-local address, enforcing bootstrap checks
[2019-11-05T12:34:28,268][INFO ][o.e.c.s.ClusterService   ] [node_1] new_master {node_1}{b7PKnpL8TBqfX_j6N0ydyQ}{8rQez1gDSturfNgSHluIyw}{192.168.16.152}{192.168.16.152:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2019-11-05T12:34:28,651][INFO ][o.e.h.n.Netty4HttpServerTransport] [node_1] publish_address {192.168.16.152:9200}, bound_addresses {[::]:9200}
[2019-11-05T12:34:28,651][INFO ][o.e.n.Node               ] [node_1] started
[2019-11-05T12:34:28,676][INFO ][o.e.g.GatewayService     ] [node_1] recovered [0] indices into cluster_state

 

至此,elasticsearch+ik分词工具安装完成!

 


  

以上是关于Elasticsearch-5.5.0以及elasticsearch-analysis-ik安装使用的主要内容,如果未能解决你的问题,请参考以下文章

elasticsearch5.5安装部署

windows 下安装elasticsearch

基于Opencv的几种立体匹配算法+ELAS

资讯Elas推出比特币企业套件Signavera

[ES]Python查询ES导出数据为Excel

配置elasticsearch 以及ik分词