无法锁定JVM内存：error = 12，reason =无法分配内存 - Elasticsearch 6.2.4

Posted

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了无法锁定JVM内存：error = 12，reason =无法分配内存 - Elasticsearch 6.2.4相关的知识，希望对你有一定的参考价值。

我正在努力提高Elasticsearch 6.2.4的性能。我正在设置bootstrap.memory_lock: true。我做了以下更改

1）文件/ etc / default / elasticsearch

ES_JAVA_OPTS="-Xms4g -Xmx4g" 
MAX_LOCKED_MEMORY=unlimited

2）文件/etc/security/limits.conf

elasticsearch soft memlock unlimited
elasticsearch hard memlock unlimited

3）文件/usr/lib/systemd/system/elasticsearch.service如下更改并运行systemctl daemon-reload

LimitMEMLOCK=infinity

4）文件/etc/elasticsearch/elasticsearch.yml

bootstrap.memory_lock: true

5）文件/etc/elasticsearch/jvm.options

-Xms4g
-Xmx4g

ulimit -as输出

core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 30689
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 30689
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

进行这些更改后，Elasticsearch停止使用以下日志

[2018-07-17T12:58:17,514][WARN ][o.e.b.JNANatives         ] Unable to 

lock JVM Memory: error=12, reason=Cannot allocate memory
[2018-07-17T12:58:17,517][WARN ][o.e.b.JNANatives         ] This can result in part of the JVM being swapped out.
[2018-07-17T12:58:17,517][WARN ][o.e.b.JNANatives         ] Increase RLIMIT_MEMLOCK, soft limit: 65536, hard limit: 65536
[2018-07-17T12:58:17,517][WARN ][o.e.b.JNANatives         ] These can be adjusted by modifying /etc/security/limits.conf, for example: 
    # allow user 'elasticsearch' mlockall
    elasticsearch soft memlock unlimited
    elasticsearch hard memlock unlimited
[2018-07-17T12:58:17,518][WARN ][o.e.b.JNANatives         ] If you are logged in interactively, you will have to re-login for the new limits to take effect.
[2018-07-17T12:58:17,684][INFO ][o.e.n.Node               ] [] initializing ...
[2018-07-17T12:58:17,757][INFO ][o.e.e.NodeEnvironment    ] [8fsU41g] using [1] data paths, mounts [[/ (/dev/nvme0n1p1)]], net usable_space [5.4gb], net total_space [7.6gb], types [ext4]
[2018-07-17T12:58:17,758][INFO ][o.e.e.NodeEnvironment    ] [8fsU41g] heap size [3.9gb], compressed ordinary object pointers [true]
[2018-07-17T12:58:17,808][INFO ][o.e.n.Node               ] node name [8fsU41g] derived from node ID [8fsU41ghScq506TqNnjegQ]; set [node.name] to override
[2018-07-17T12:58:17,809][INFO ][o.e.n.Node               ] version[6.2.4], pid[2823], build[ccec39f/2018-04-12T20:37:28.497551Z], OS[Linux/4.4.0-1062-aws/amd64], JVM[Oracle Corporation/Java HotSpot(TM) 64-Bit Server VM/1.8.0_171/25.171-b11]
[2018-07-17T12:58:17,809][INFO ][o.e.n.Node               ] JVM arguments [-Xms4g, -Xmx4g, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=75, -XX:+UseCMSInitiatingOccupancyOnly, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Djava.io.tmpdir=/tmp/elasticsearch.POxZWZQp, -XX:+HeapDumpOnOutOfMemoryError, -XX:HeapDumpPath=/var/lib/elasticsearch, -XX:+PrintGCDetails, -XX:+PrintGCDateStamps, -XX:+PrintTenuringDistribution, -XX:+PrintGCApplicationStoppedTime, -Xloggc:/var/log/elasticsearch/gc.log, -XX:+UseGCLogFileRotation, -XX:NumberOfGCLogFiles=32, -XX:GCLogFileSize=64m, -Xms4g, -Xmx4g, -Des.path.home=/usr/share/elasticsearch, -Des.path.conf=/etc/elasticsearch]
[2018-07-17T12:58:18,564][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [aggs-matrix-stats]
[2018-07-17T12:58:18,564][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [analysis-common]
[2018-07-17T12:58:18,564][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [ingest-common]
[2018-07-17T12:58:18,564][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [lang-expression]
[2018-07-17T12:58:18,564][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [lang-mustache]
[2018-07-17T12:58:18,564][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [lang-painless]
[2018-07-17T12:58:18,564][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [mapper-extras]
[2018-07-17T12:58:18,565][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [parent-join]
[2018-07-17T12:58:18,565][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [percolator]
[2018-07-17T12:58:18,565][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [rank-eval]
[2018-07-17T12:58:18,565][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [reindex]
[2018-07-17T12:58:18,565][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [repository-url]
[2018-07-17T12:58:18,565][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [transport-netty4]
[2018-07-17T12:58:18,565][INFO ][o.e.p.PluginsService     ] [8fsU41g] loaded module [tribe]
[2018-07-17T12:58:18,565][INFO ][o.e.p.PluginsService     ] [8fsU41g] no plugins loaded
[2018-07-17T12:58:21,149][INFO ][o.e.d.DiscoveryModule    ] [8fsU41g] using discovery type [zen]
[2018-07-17T12:58:21,633][INFO ][o.e.n.Node               ] initialized
[2018-07-17T12:58:21,633][INFO ][o.e.n.Node               ] [8fsU41g] starting ...
[2018-07-17T12:58:21,767][INFO ][o.e.t.TransportService   ] [8fsU41g] publish_address {172.31.20.225:9300}, bound_addresses {[::]:9300}
[2018-07-17T12:58:21,790][INFO ][o.e.b.BootstrapChecks    ] [8fsU41g] bound or publishing to a non-loopback address, enforcing bootstrap checks
[2018-07-17T12:58:21,792][ERROR][o.e.b.Bootstrap          ] [8fsU41g] node validation exception
[1] bootstrap checks failed
[1]: memory locking requested for elasticsearch process but memory is not locked
[2018-07-17T12:58:21,794][INFO ][o.e.n.Node               ] [8fsU41g] stopping ...
[2018-07-17T12:58:21,820][INFO ][o.e.n.Node               ] [8fsU41g] stopped
[2018-07-17T12:58:21,820][INFO ][o.e.n.Node               ] [8fsU41g] closing ...
[2018-07-17T12:58:21,832][INFO ][o.e.n.Node               ] [8fsU41g] closed

我是否应该加入任何其他更改以使其正常工作？

答案

跑：

sudo vim /usr/lib/systemd/system/elasticsearch.service

组：

[Service]
LimitMEMLOCK=infinity

添加[Service]解决了我的问题。

另一答案

你输出的ulimit

max locked memory       (kbytes, -l) 64

会建议/etc/security/limits.conf文件中的设置无效。尝试停止用户弹性搜索的所有会话（注销用户，停止服务，也许最后重启机器）。

我有类似的问题，只有通过添加LimitMEMLOCK=infinity（列表中的第3步）才能解决，不要忘记在将systemctl daemon-reload添加到文件/usr/lib/systemd/system/elasticsearch.service后运行它

另一答案

必须重新加载服务？

sudo / bin / systemctl daemon-reload

sudo systemctl restart elasticsearch.service sudo systemctl status elasticsearch.service

另一答案

在/usr/lib/systemd/system/elasticsearch.service;在[服务]下定义“MAX_LOCKED_MEMORY =无限制” - 我错误地将设置添加到文件的末尾，并获得与您相同的输出 - 但将其移动到文件的右侧部分，并运行“service elasticsearch start” “为我解决了

以上是关于无法锁定JVM内存：error = 12，reason =无法分配内存 - Elasticsearch 6.2.4的主要内容，如果未能解决你的问题，请参考以下文章

IDEA无法启动：Failed to create JVM：error code -4

无法在 SQL Server Express 版本中启用内存中的锁定页面

JAVA无法创建大于1G内存的JVM

JVM-Java内存模型-20200217

为啥操作系统说内存足够时无法为jvm分配内存

java异常（转）