Elasticsearch跨集群搜索(Cross Cluster Search)

Posted luxiaoxun

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了Elasticsearch跨集群搜索(Cross Cluster Search)相关的知识,希望对你有一定的参考价值。

1、简介

Elasticsearch在5.3版本中引入了Cross Cluster Search(CCS 跨集群搜索)功能,用来替换掉要被废弃的Tribe Node。类似Tribe Node,Cross Cluster Search用来实现跨集群的数据搜索。

2、配置Cross Cluster Search

假设我们有2个ES集群:

Node
Address
Port
Transport Port
Cluster
elasticsearch01
127.0.0.1
9201
9301
America
elasticsearch02
127.0.0.1
9202
9302
America
elasticsearch03
127.0.0.1
9203
9303
Europe
elasticsearch04
127.0.0.1
9204
9304
Europe

有2种方式可以用来配置CCS:

1)配置elasticsearch.yml

search:
    remote:
        america:
            seeds: 127.0.0.1:9301
            seeds: 127.0.0.1:9302
        europe:
            seeds: 127.0.0.1:9303
            seeds: 127.0.0.1:9304

注意:以上方式,在配置的时候,需要remote cluster处在运行状态。比如在配置“america”的集群的时候,需要“europe”集群处在运行状态,否则节点无法启动成功。

2)使用 Cluster Settings API配置

curl -XPUT -H‘Content-Type: application/json‘ localhost:9201/_cluster/settings -d ‘
{
    "persistent": {
        "search.remote": {
            "america": {
                "skip_unavailable": "true",
                "seeds": ["127.0.0.1:9301","127.0.0.1:9302"]
            },
            "europe": {
                "skip_unavailable": "true",
                "seeds": ["127.0.0.1:9303","127.0.0.1:9304"]
            }
        }
    }
}‘

推荐使用API方式,可以方便的修改remote cluster的seeds和其他配置。

3、验证Cross Cluster Search

1)使用_remote/info查看CCS连接状态:

[[email protected] elasticsearch01]# curl -XGET -H ‘Content-Type: application/json‘ localhost:9201/_remote/info?pretty
{
  "america" : {
    "seeds" : [
      "127.0.0.1:9301",
      "127.0.0.1:9302"
    ],
    "http_addresses" : [
      "127.0.0.1:9201",
      "127.0.0.1:9202"
    ],
    "connected" : true,
    "num_nodes_connected" : 2,
    "max_connections_per_cluster" : 3,
    "initial_connect_timeout" : "30s"
  },
  "europe" : {
    "seeds" : [
      "127.0.0.1:9303",
      "127.0.0.1:9304"
    ],
    "http_addresses" : [
      "127.0.0.1:9203",
      "127.0.0.1:9204"
    ],
    "connected" : true,
    "num_nodes_connected" : 2,
    "max_connections_per_cluster" : 3,
    "initial_connect_timeout" : "30s"
  }
}

2)使用跨集群搜索:

同时查询2个集群的数据:

GET /cluster_name:index,cluster_name:index/_search
GET */index/_search

java API 示例:

//查询所有集群,以appIndex-开头的数据
SearchRequest searchRequest = Requests.searchRequest("*:appIndex-*");
SearchResponse response = es.getClient().search(searchRequest).get();

4、Disable Cross Cluster Search

使用API设置:

curl -XPUT -H‘Content-Type: application/json‘ localhost:9201/_cluster/settings -d ‘
{
    "persistent": {
        "search.remote": {
            "america": {
                "skip_unavailable": null,
                "seeds": null
            },
            "europe": {
                "skip_unavailable": null,
                "seeds": null
            }
        }
    }
}‘

5、CCS的配置

search.remote.${cluster_alias}.skip_unavailable:查询的时候skip不可达的集群,默认false,推荐设置为true

search.remote.connect:默认true,即任何node都作为一个cross-cluster client去连接remote cluster,跨集群搜索的请求必须发给cross-cluster client。

search.remote.node.attr:设置remote node的属性,比如search.remote.node.attr:gateway这样设置,只有node.attr.gateway: true的node才会被其他的cross-cluster client连接用来做CCS查询。

 

参考:

https://www.elastic.co/guide/en/elasticsearch/reference/6.2/modules-cross-cluster-search.html

http://kelonsoftware.com/elasticsearch-cross-cluster-search/

 

以上是关于Elasticsearch跨集群搜索(Cross Cluster Search)的主要内容,如果未能解决你的问题,请参考以下文章

Elasticsearch:跨集群搜索 Cross-cluster search(CCS)及安全

elasticsearch跨集群搜索

Elasticsearch:如何为 CCR 及 CCS 建立带有安全的集群之间的互信

Elasticsearch 主从同步之跨集群复制

Elasticsearch学习之深入搜索四 --- cross-fields搜索

elasticsearch 跨网段组集群