elasticsearch 写入优化

Posted 2021-01-19

全量dump数据时，为优化性能，可做如下优化。

1. 分片设置，不分片

http://localhost:9200/test_index/_settings/
{
  "index": {
    "number_of_replicas": 0
  }
}

2. 刷新设置，不刷新

http://localhost:9200/test_index/_settings/
{
  "index": {
    "refresh_interval": "-1"
  }
}

3. translog 大小设置，调大 默认512M

http://localhost:9200/test_index/_settings/
{
  "index.translog.flush_threshold_size": "1024mb"
}

4. 如果是SSD硬盘，修改段合并速率

http://localhost:9200/_cluster/settings/
{
  "persistent": {
    "indices.store.throttle.max_bytes_per_sec": "200mb"
  }
}

http://localhost:9200/_cluster/settings/
{
  "transient": {
    "indices.store.throttle.type": "none"
  }
}

全量dump后，再恢复一下

1. 分片设置

http://localhost:9200/test_index/_settings/
{
  "index": {
    "number_of_replicas": 5
  }
}

2. 刷新设置，不刷新

http://localhost:9200/test_index/_settings/
{
  "index": {
    "refresh_interval": "1s"
  }
}

3. translog 大小设置

http://localhost:9200/test_index/_settings/
{
  "index.translog.flush_threshold_size": "512mb"
}

当然应该使用bulk模式， elasticsearch单次提交数据尽量在15M以内。