Solr-4-集群安装

Posted 2021-05-02 健哥说编程

1、Solr集群安装

使用Solr自带的Jetty服务器+Zookeeper配置Solr集群。

 

装备工作：

 1：Linux CentOS7 3台或3台虚拟机。

 2：JDK1.8安装包Linux-x64.gz

 3：zookeeper-3.4.15.tar.gz安装包

 4：Solr7.5-tar.gz安装包。

 5：开发两个脚本

1）使用ssh远程执行所有命令

#!/bin/bash

usage="用法:$0 hosts.txt  \"command"\"

if [ $# -lt 2 ]; then

    echo ${usage}

    exit 1

fi

file=$1

cmd=$2

for host in `cat $file`

do

    script="ssh ${host} ${cmd}"

    echo ${script}

    eval ${script}

done

exit 0

 

2）使用scp进行网络复制

#!/bin/bash

usage="用法:$0 hosts.sh src destDir"

if [ $# -lt 3 ];then

    echo $usage

    exit 1

fi

 

for host in `cat $1`

do

    script="scp -r $2 $host:$3"

    echo $script

    eval $script

done

exit 0

 

 

步1：规划

服务器	软件	进程
192.168.56.41/server41	Zookeeper Solr	QuormPeerMan Start.jar(Solr)
192.168.56.42/server42	Zookeeper Solr	QuormPeerMan Start.jar(Solr)
192.168.56.43/server43	Zookeeper Solr	QuormPeerMan Start.jar(Solr)

 

步2：Linux基本设置

1、关闭所有主机防火墙

# ./cmd.sh hosts.txt  “systemctl stop firewalld”

2、禁用所有主机的防火墙

#./cmd.sh hosts.txt “systemctl disable firewalld”

3、在所有主机上安装JDk并设置环境变量，然后使用scp同步所有主机。

4、修改主机的机器名（根据上表要求）、修改/etc/hosts配置文件。同步的其他主机。

5、配置ssh免密码登录，从server41到其他的两台主机。

 

步3、安装zookeeper

1、创建目录 #mkdir /app

2、上传zookeeper

3、解压zookeeper , # tar -zxvf ~/zookeeper-3.4.13.tar.gz -C /app/

4、配置zookeeper

$ZOOKEEPER_HOME/conf/zoo.cfg

配置以下内容：

dataDir=/app/zookeeper/data

server.41=server41:2888:3888

server.42=server42:2888:3888

server.43=server43:2888:3888

5、在每一个服务器的/app/zookeeper/data目录下，创建一个myid里面是这个zookeeper的id。如server.41配置的id为41，即 echo 41 > /app/zookeeper/data/myid。其他类似。

6、配置zookeeper环境变量

export ZK_HOME=/app/zookeeper-3.4.13

export PATH=$PATH:$ZK_HOME/bin

7、现在启动zookeeper使用命令：./cmd.sh hosts.sh “zkServer.sh start”

8、查看zookeeper的状态，执行命令：./cmd.sh hosts.sh “zkServer.sh status”

[root@server41 ~]# ./cmd.sh hosts.txt "zkServer.sh status"

ssh server41 zkServer.sh status

ZooKeeper JMX enabled by default

Using config: /app/zookeeper-3.4.13/bin/../conf/zoo.cfg

Mode: follower

ssh server42 zkServer.sh status

ZooKeeper JMX enabled by default

Using config: /app/zookeeper-3.4.13/bin/../conf/zoo.cfg

Mode: leader

ssh server43 zkServer.sh status

ZooKeeper JMX enabled by default

Using config: /app/zookeeper-3.4.13/bin/../conf/zoo.cfg

Mode: follower

如上所示，如果有的显示了leader有的显示了follower则为启动成功。

 

步4、安装Solr

1) 、上传并解压Solr

# tar -zxvf solr-7.5.0.tar.gz -C /app/

2) 、配置solr.in.sh

在SOLR_HOME/bin/solr.in.sh中配置如下：

ZK_HOST="server41:2181,server42:2181,server43:2181"

SOLR_HOST="server41"

 

注意：

可以将ZK_HOST设置为一个zookeeper的子目录如下：

              ZK_HOST=”server41:2181,server42:2181,server43:2181/solr”注意后面的/solr子目录，可以是任意的目录。之前请先通过zkCli.sh登录以后，创建这个目录，即：

zkCli.sh

[zookeeper]mkdir /solr

其后再启动solr集群查看，此时所有solr的配置信息，都将会在zookeeper:2181/solr这个子目录下：

[zk: localhost:2181(CONNECTED) 0] ls /

[zookeeper, solr]

[zk: localhost:2181(CONNECTED) 1] ls /solr

[configs, overseer, aliases.json, live_nodes, collections, overseer_elect, security.json, clusterstate.json, autoscaling, autoscaling.json]

[zk: localhost:2181(CONNECTED) 2] ls /solr/configs

[_default] 这儿将默认上传一个配置文件目录。

 

 

3) 、配置solr.xml文件

在/server/solr目录下，修改solr.xml文件如下：

<str name="host">${host:server41}</str>

即在host后面，添加主机的名称。每机主机，等于当前主机的名称。

4) 、启动solr

配置好以后，将Solr分发到其他的主机，并修改不同的地方。然后启动Solr。

 

5) 、启动solr,如果都配置了环境变量，可以直接使用以下方式启动

# ./cmd.sh hosts.txt “solr start -force”

注意，如果没有配置ZK_HOST则默认启动为非集群模式。可以通过以下方式在启动时指定zookeeper:

#solr start -cloud -z server41:2181,server42:2181,server43:2181 -force

 

6）、可选的配置环境变量

 

export SOLR_BASE=/app/solr-7.5.0

export SOLR_HOME=/app/solr-7.5.0/server/solr

export PATH=$PATH:$SOLR_BASE/bin

注意，上面的SOLR_HOME目录下，必须要拥有solr.xml文件。否则启动失败。

 

 

 

步5、查看状态

如果查询Cloud/Nodes发现有三个节点，则为配置成功。

 

 

另：其他启动方式，也可以在启动时

 

 

查看Solr状态：

[root@server43 bin]# ./solr status

Found 1 Solr nodes:

Solr process 1551 running on port 8983

INFO  - 2018-10-05 23:29:35.162; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop

{

  "solr_home":"/app/solr-7.5.0/server/solr",

  "version":"7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55",

  "startTime":"2018-10-05T14:51:02.732Z",

  "uptime":"0 days, 0 hours, 38 minutes, 33 seconds",

  "memory":"55.2 MB (%11.3) of 490.7 MB",

  "cloud":{

    "ZooKeeper":"server41:2181,server42:2181,server43:2181",

    "liveNodes":"3",

"collections":"0"}}

 

查看zookeeper上的节点数据：

[zk: localhost:2181(CONNECTED) 6] ls /

[configs, zookeeper, overseer, aliases.json, live_nodes, collections, overseer_elect, security.json, clusterstate.json, autoscaling, autoscaling.json]

 

 

步6、停止Solr集群

[root@server41 ~]# ./cmd.sh hosts.txt "/app/solr-7.5.0/bin/solr stop"

ssh server41 /app/solr-7.5.0/bin/solr stop

Sending stop command to Solr running on port 8983 ... waiting up to 180 seconds to allow Jetty process 1680 to stop gracefully.

ssh server42 /app/solr-7.5.0/bin/solr stop

Sending stop command to Solr running on port 8983 ... waiting up to 180 seconds to allow Jetty process 1596 to stop gracefully.

ssh server43 /app/solr-7.5.0/bin/solr stop

Sending stop command to Solr running on port 8983 ... waiting up to 180 seconds to allow Jetty process 1551 to stop gracefully.

 

 

步7、创建Collection

重新启动集群，然后，创建一个Collection。

先启动zk集群：

# ./cmd.sh hosts.txt "zkServer.sh start"

再启动Solr集群

# ./cmd.sh hosts.txt "/app/solr-7.5.0/bin/solr start -cloud -force"

查看状态：

[root@server41 ~]# solr status

Found 1 Solr nodes:

Solr process 1392 running on port 8983

INFO  - 2018-10-06 21:40:22.083; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop

{

  "solr_home":"/app/solr-7.5.0/server/solr",

  "version":"7.5.0 b5bf70b7e32d7ddd9742cc821d471c5fabd4e3df - jimczi - 2018-09-18 13:07:55",

  "startTime":"2018-10-06T13:28:25.577Z",

  "uptime":"0 days, 0 hours, 11 minutes, 57 seconds",

  "memory":"35.2 MB (%7.2) of 490.7 MB",

  "cloud":{

    "ZooKeeper":"server41:2181,server42:2181,server43:2181",

    "liveNodes":"3",

    "collections":"1"}}

 

 

创建：

[root@server41 ~]# solr create_collection -c collection1 -shards 2 -replicationFactor 3 -force

WARNING: Using _default configset with data driven schema functionality. NOT RECOMMENDED for production use.

         To turn off: bin/solr config -c collection1 -p 8983 -action set-user-property -property update.autoCreateFields -value false

INFO  - 2018-10-06 21:39:30.792; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop

Created collection 'collection1' with 2 shard(s), 3 replica(s) with config-set 'collection1'

出现最后一句说明创建成功。

-c 指定库(collection)名称

-shards 指定分片数量,可简写为 -s ,索引数据会分布在这些分片上

-replicationFactor 每个分片的副本数量,每个分片由至少1个物理副本组成

 

步8、查看collection状态

 

查看网8983端口

 

 

查看两个Shade:

 

 

通过命令查看状态这个Connection的状态：

[root@server41 ~]# solr healthcheck -c collection1

INFO  - 2018-10-06 21:52:41.806; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop

{

  "collection":"collection1",

  "status":"healthy",

  "numDocs":0,

  "numShards":2,

  "shards":[

    {

      "shard":"shard1",

      "status":"healthy",

      "replicas":[

        {

          "name":"core_node3",

          "url":"http://server42:8983/solr/collection1_shard1_replica_n1/",

          "numDocs":0,

          "status":"active",

          "uptime":"0 days, 0 hours, 24 minutes, 7 seconds",

          "memory":"38.3 MB (%7.8) of 490.7 MB"},

        {

          "name":"core_node5",

          "url":"http://server41:8983/solr/collection1_shard1_replica_n2/",

          "numDocs":0,

          "status":"active",

          "uptime":"0 days, 0 hours, 24 minutes, 17 seconds",

          "memory":"76.9 MB (%15.7) of 490.7 MB"},

        {

          "name":"core_node7",

          "url":"http://server43:8983/solr/collection1_shard1_replica_n4/",

          "numDocs":0,

          "status":"active",

          "uptime":"0 days, 0 hours, 23 minutes, 56 seconds",

          "memory":"69.5 MB (%14.2) of 490.7 MB",

          "leader":true}]},

    {

      "shard":"shard2",

      "status":"healthy",

      "replicas":[

        {

          "name":"core_node9",

          "url":"http://server42:8983/solr/collection1_shard2_replica_n6/",

          "numDocs":0,

          "status":"active",

          "uptime":"0 days, 0 hours, 24 minutes, 7 seconds",

          "memory":"38.3 MB (%7.8) of 490.7 MB"},

        {

          "name":"core_node11",

          "url":"http://server41:8983/solr/collection1_shard2_replica_n8/",

          "numDocs":0,

          "status":"active",

          "uptime":"0 days, 0 hours, 24 minutes, 17 seconds",

          "memory":"77.5 MB (%15.8) of 490.7 MB"},

        {

          "name":"core_node12",

          "url":"http://server43:8983/solr/collection1_shard2_replica_n10/",

          "numDocs":0,

          "status":"active",

          "uptime":"0 days, 0 hours, 23 minutes, 56 seconds",

          "memory":"70.3 MB (%14.3) of 490.7 MB",

          "leader":true}]}]}

 

 

 

步9、上传配置文件

上传本地的默认配置文件，到zookeeper集群：

[root@server43 app]# solr zk upconfig -d /app/solr-7.5.0/server/solr/configsets/_default -n _default -z server41:2181,server42:2181,server43:2181

INFO  - 2018-10-06 23:13:43.886; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop

Uploading /app/solr-7.5.0/server/solr/configsets/_default/conf for config _default to ZooKeeper at server41:2181,server42:2181,server43:2181

上传完成以后，登录zookeeper查看，存在_default这个配置：

[zk: localhost:2181(CONNECTED) 2] ls /configs

[collection1,_default]

 

删除配置文件

[root@server43 app]# solr zk rm -r /configs/_default -z server41:2181

INFO  - 2018-10-06 23:18:59.494; org.apache.solr.util.configuration.SSLCredentialProviderFactory; Processing SSL Credential Provider chain: env;sysprop

Removing Zookeeper node /configs/_default from ZooKeeper at server41:2181 recurse: true

 

删除完成以后，登录zookeeper查看，已经没有这个数据了：

[zk: localhost:2181(CONNECTED) 2] ls /configs

[collection1]