elasticsearch的两个异常

Posted 2020-08-20 jiu__

1 _riverStatus Import_fail 

问题描述： 发现有个索引的数据同步不完整，在 http://192.168.1.17:9200/_plugin/head/ 在browse - river里看到 _riverStatus Import_fail

查看 elasticsearch 的log发现 有几条数据由于异常造成同步失败，处理好数据好重新建索引数据同步正常。

2 EsRejectedExecutionException[rejected execution (queue capacity 50) on

此异常主要是因为 bulk thead pool set  queue capacity =50 这个可以设置大点

打开 elasticsearch.yml 在末尾加上 

threadpool:
    bulk:
        type: fixed
        size: 60
        queue_size: 1000

重新启动服务即可

另：

--查看线程池设置--
curl -XGET "http://localhost:9200/_nodes/thread_pool/"

类似thread pool设置均可以这样修改
threadpool:
    index:
        type: fixed
        size: 30
        queue_size: 1000

 

关于第二个异常，有网友给的解释挺好的：

Elasticsearch has a thread pool and a queue for search per node. A thread pool will have N number of workers ready to handle the requests. When a request comes and if a worker is free , this is handled by the worker. Now by default the number of workers is equal to the number of cores on that CPU. When the workers are full and there are more search requests , the request will go to queue. The size of queue is also limited. Its by default size is say 100 and if there happens more parallel requests than this , then those requests would be rejected as you can see in the error log.

The solution to this would be to -

1.    Increase the size of queue or threadpool - The immediate solution for this would be to increase the size of the search queue. We can also increase the size of threadpool , but then that might badly effect the performance of individual queries. So increasing the queue might be a good idea. But then remember that this queue is memory residential and increasing the queue size too much can result in Out Of Memory issues. You can get more info on the same here.
2.    Increase number of nodes and replicas - Remember each node has its own search threadpool/queue. Also search can happen on primary shard OR replica.

关于thread pool，可参看：https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-threadpool.html