ELK踩过的各种坑 6.4版本

Posted 2021-01-03 suffergtf

一、elasticsearch

1、服务正常启动，但不能正常访问

[[email protected]node1 elasticsearch]# systemctl start elasticsearch
[r[email protected]-node1 elasticsearch]# systemctl status elasticsearch
● elasticsearch.service - Elasticsearch
   Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; disabled; vendor preset: disabled)
   Active: active (running) since 五 2018-08-31 10:44:45 CST; 5s ago
     Docs: http://www.elastic.co
 Main PID: 1627 (java)
   CGroup: /system.slice/elasticsearch.service
           ├─1627 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCM...
           └─1680 /usr/share/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller

8月 31 10:44:45 linux-node1 systemd[1]: Started Elasticsearch.
8月 31 10:44:45 linux-node1 systemd[1]: Starting Elasticsearch...
8月 31 10:44:46 linux-node1 elasticsearch[1627]: Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMar...ase.
Hint: Some lines were ellipsized, use -l to show in full

日志信息
[[email protected] ~]# cat /var/log/elasticsearch/oldboy.log 
[2018-08-31T11:29:02,215][WARN ][o.e.b.JNANatives         ] Unable to lock JVM Memory: error=12, reason=无法分配内存
[2018-08-31T11:29:02,218][WARN ][o.e.b.JNANatives         ] This can result in part of the JVM being swapped out.
[2018-08-31T11:29:02,219][WARN ][o.e.b.JNANatives         ] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
[2018-08-31T11:29:02,219][WARN ][o.e.b.JNANatives         ] These can be adjusted by modifying /etc/security/limits.conf, for example: 
    # allow user ‘elasticsearch‘ mlockall
    elasticsearch soft memlock unlimited
    elasticsearch hard memlock unlimited
[2018-08-31T11:29:02,219][WARN ][o.e.b.JNANatives         ] If you are logged in interactively, you will have to re-login for the new limits to take effect.
[2018-08-31T11:29:02,844][INFO ][o.e.n.Node               ] [linux-node1] initializing ...
[2018-08-31T11:29:03,036][INFO ][o.e.e.NodeEnvironment    ] [linux-node1] using [1] data paths, mounts [[/ (rootfs)]], net usable_space [15.3gb], net total_space [17.7gb], types [rootfs]
[2018-08-31T11:29:03,036][INFO ][o.e.e.NodeEnvironment    ] [linux-node1] heap size [1015.6mb], compressed ordinary object pointers [true]
[2018-08-31T11:29:03,038][INFO ][o.e.n.Node               ] [linux-node1] node name [linux-node1], node ID [8HbHSciQTHqiNpgfM2osEQ]
[2018-08-31T11:29:03,038][INFO ][o.e.n.Node               ] [linux-node1] version[6.4.0], pid[1003], build[default/rpm/595516e/2018-08-17T23:18:47.308994Z], OS[Linux/3.10.0-514.el7.x86_64/amd64], JVM["Oracle Corporation"/Java HotSpot(TM) 64-Bit Server VM/10.0.1/10.0.1+10]
[2018-08-31T11:29:03,038][INFO ][o.e.n.Node               ] [linux-node1] JVM arguments [-Xms1g, -Xmx1g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.lXjdTISW, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:ErrorFile=/var/log/elasticsearch/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/var/log/elasticsearch/gc.log:utctime,pid,tags:filecount=32,filesize=64m, -Djava.locale.providers=COMPAT, -XX:UseAVX=2, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch, -Des.distribution.flavor=default, -Des.distribution.type=rpm]
[2018-08-31T11:29:07,406][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [aggs-matrix-stats]
[2018-08-31T11:29:07,406][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [analysis-common]
[2018-08-31T11:29:07,407][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [ingest-common]
[2018-08-31T11:29:07,407][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [lang-expression]
[2018-08-31T11:29:07,407][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [lang-mustache]
[2018-08-31T11:29:07,407][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [lang-painless]
[2018-08-31T11:29:07,407][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [mapper-extras]
[2018-08-31T11:29:07,407][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [parent-join]
[2018-08-31T11:29:07,407][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [percolator]
[2018-08-31T11:29:07,407][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [rank-eval]
[2018-08-31T11:29:07,407][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [reindex]
[2018-08-31T11:29:07,407][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [repository-url]
[2018-08-31T11:29:07,407][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [transport-netty4]
[2018-08-31T11:29:07,408][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [tribe]
[2018-08-31T11:29:07,408][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [x-pack-core]
[2018-08-31T11:29:07,408][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [x-pack-deprecation]
[2018-08-31T11:29:07,408][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [x-pack-graph]
[2018-08-31T11:29:07,408][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [x-pack-logstash]
[2018-08-31T11:29:07,408][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [x-pack-ml]
[2018-08-31T11:29:07,409][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [x-pack-monitoring]
[2018-08-31T11:29:07,409][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [x-pack-rollup]
[2018-08-31T11:29:07,409][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [x-pack-security]
[2018-08-31T11:29:07,409][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [x-pack-sql]
[2018-08-31T11:29:07,410][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [x-pack-upgrade]
[2018-08-31T11:29:07,410][INFO ][o.e.p.PluginsService     ] [linux-node1] loaded module [x-pack-watcher]
[2018-08-31T11:29:07,410][INFO ][o.e.p.PluginsService     ] [linux-node1] no plugins loaded
[2018-08-31T11:29:18,133][INFO ][o.e.x.s.a.s.FileRolesStore] [linux-node1] parsed [0] roles from file [/etc/elasticsearch/roles.yml]
[2018-08-31T11:29:19,562][INFO ][o.e.x.m.j.p.l.CppLogMessageHandler] [controller/1056] [[email protected]] controller (64 bit): Version 6.4.0 (Build cf8246175efff5) Copyright (c) 2018 Elasticsearch BV
[2018-08-31T11:29:20,568][DEBUG][o.e.a.ActionModule       ] Using REST wrapper from plugin org.elasticsearch.xpack.security.Security
[2018-08-31T11:29:21,310][INFO ][o.e.d.DiscoveryModule    ] [linux-node1] using discovery type [zen]
[2018-08-31T11:29:24,639][INFO ][o.e.n.Node               ] [linux-node1] initialized
[2018-08-31T11:29:24,640][INFO ][o.e.n.Node               ] [linux-node1] starting ...
[2018-08-31T11:29:25,457][INFO ][o.e.t.TransportService   ] [linux-node1] publish_address {192.168.127.201:9300}, bound_addresses {[::]:9300}
[2018-08-31T11:29:25,526][INFO ][o.e.b.BootstrapChecks    ] [linux-node1] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2018-08-31T11:29:25,544][ERROR][o.e.b.Bootstrap          ] [linux-node1] node validation exception
[1] bootstrap checks failed
[1]: memory locking requested for elasticsearch process but memory is not locked
[2018-08-31T11:29:25,560][INFO ][o.e.n.Node               ] [linux-node1] stopping ...
[2018-08-31T11:29:25,627][INFO ][o.e.n.Node               ] [linux-node1] stopped
[2018-08-31T11:29:25,628][INFO ][o.e.n.Node               ] [linux-node1] closing ...
[2018-08-31T11:29:25,681][INFO ][o.e.n.Node               ] [linux-node1] closed
[2018-08-31T11:29:25,686][INFO ][o.e.x.m.j.p.NativeController] Native controller process has stopped - no new native processes can be started

处理
[[email protected] ~]# vim /etc/systemd/system.conf　　　　#####修改如下行，即可解决error报错
DefaultLimitNOFILE=65536
DefaultLimitNPROC=32000
DefaultLimitMEMLOCK=infinity

[[email protected] elasticsearch]# vim /etc/security/limits.conf　　　　#####添加如下行，即可解决warn警告

# allow user ‘elasticsearch‘ mlockall
elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited