Redis三种集群模式之redis-cluster模式

Posted 2022-02-12 Locutus

1. redis-cluster模式

之前有看过redis集群部署的三种方案，不过性能最高的还是redis官方推荐的redis-cluster模式，下面介绍一下redis-cluster这种模式。

a)采用去中心化的思想，没有中心节点的说法，它使用hash slot方式将16348个hash slot覆盖到所有节点上，对于存储的每个key值，使用CRC16（KEY）&16348=slot得到他对应的hash slot，

并在访问key的时候就去找他的hash slot在哪一个节点上，然后由当前访问节点从实际被分配了这个hash slot的节点去取数据，节点之间使用轻量协议通信 减少带宽占用 性能很高，

自动实现负载均衡与高可用，自动实现failover并且支持动态扩展。

b)其内部中也需要配置主从，并且内部也是采用哨兵模式，如果有半数节点发现某个异常节点，共同决定更改异常节点的状态，如果改节点是主节点，则对应的从节点自动顶替为主节点，当原先的主节点上线后，则会变为从节点。

如果集群中的master没有slave节点，则master挂掉后整个集群就会进入fail状态，因为集群的slot映射不完整。如果集群超过半数以上的master挂掉，无论是否有slave，集群都会进入fail状态。

c)根据官方推荐，集群部署至少要3台以上的master节点。

2. redis-cluster模式集群配置

创建目录

# cd /usr/software/redis/redis-cluster/

# mkdir -p /usr/software/redis/redis-cluster/7000, 7001, 7002, 7003, 7004, 7005

# mkdir -p /usr/software/redis/redis-cluster/7000/log, data
...
...
...

在每一个节点路径下，修改一下配置文件redis.conf

# cd /usr/software/redis/redis-cluster/7000

# vim redis.conf

修改如下键值对

####### NETWORK #####################################
bind 192.168.10.101
protected-mode yes
port 7000
####### GENERAL ######################################
daemonize yes
pidfile "/var/run/redis_7000.pid"
logfile "/mnt/e/redis-cluster-new/7000/log/redis.log"
####### SNAPSHOTTING  ################################
dir "/mnt/e/redis-cluster-new/7000/data"
####### REDIS CLUSTER  ###############################
cluster-enabled yes
cluster-config-file nodes-7000.conf
cluster-node-timeout 15000
####### SECURITY ###################################
requirepass "fd43W2%3@-=pi"

其他几个实例的配置文件，修改以上配置即可。

3. 集群启动

将所有redis-cluster节点启动

# cd /mnt/e/redis-cluster-new/redis-5.0.9

# nohup ./redis-server  /mnt/e/redis-cluster-new/7000/redis.conf &

# 依次启动其他redis节点

# ps -ef | grep redis
root       349   258  0 00:28 ?        00:00:08 ./redis-server 192.168.10.101:7000 [cluster]
root       359   258  0 00:28 ?        00:00:08 ./redis-server 192.168.10.101:7002 [cluster]
root       364   258  1 00:28 ?        00:00:11 ./redis-server 192.168.10.101:7003 [cluster]
root       369   258  1 00:28 ?        00:00:12 ./redis-server 192.168.10.101:7004 [cluster]
root       374   258  0 00:28 ?        00:00:08 ./redis-server 192.168.10.101:7005 [cluster]
root      2221   258  1 00:44 ?        00:00:00 ./redis-server 192.168.10.101:7001 [cluster]
root      2235   323  0 00:44 pts/1    00:00:00 grep --color=auto redis

创建redis集群。创建集群命令：其中--cluster-replicas 1代表 一个master后有几个slave，1代表为1个slave节点

# cd /mnt/e/redis-cluster-new/redis-5.0.9

# redis-cli --cluster create 192.168.10.101:7000  192.168.10.101:7001 192.168.10.101:7002 192.168.10.101:7003 192.168.10.101:7004 192.168.10.101:7005 --cluster-replicas 1 -a fd43W2%3@-=pi

过程中会提示以下内容，输入 yes 继续。

Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 192.168.10.101:7004 to 192.168.10.101:7000
Adding replica 192.168.10.101:7005 to 192.168.10.101:7001
Adding replica 192.168.10.101:7003 to 192.168.10.101:7002
>>> Trying to optimize slaves allocation for anti-affinity
[WARNING] Some slaves are in the same host as their master
M: 83508475cd7d3e783ca680bdd3d5fbae21e7e7cb 192.168.10.101:7000
   slots:[0-5460] (5461 slots) master
M: da98d89a1eb8f1c436cbe768e066ddf38cd78d28 192.168.10.101:7001
   slots:[5461-10922] (5462 slots) master
M: c6fbf1a8eead3eca8543ed8a6dc9ea38085edbef 192.168.10.101:7002
   slots:[10923-16383] (5461 slots) master
S: 5faca3ab4336233c5613b3da0017133315051b52 192.168.10.101:7003
   replicates c6fbf1a8eead3eca8543ed8a6dc9ea38085edbef
S: b6db58366b01af1868be50c23574a1f765a7b3bb 192.168.10.101:7004
   replicates 83508475cd7d3e783ca680bdd3d5fbae21e7e7cb
S: 0e49af6708619021b88b6cc862f752849c337b7e 192.168.10.101:7005
   replicates da98d89a1eb8f1c436cbe768e066ddf38cd78d28
Can I set the above configuration? (type 'yes' to accept): yes

集群自动分配结果如下：

>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
....
>>> Performing Cluster Check (using node 192.168.10.101:7000)
M: 83508475cd7d3e783ca680bdd3d5fbae21e7e7cb 192.168.10.101:7000
   slots:[0-5460] (5461 slots) master
   1 additional replica(s)
S: b6db58366b01af1868be50c23574a1f765a7b3bb 192.168.10.101:7004
   slots: (0 slots) slave
   replicates 83508475cd7d3e783ca680bdd3d5fbae21e7e7cb
S: 0e49af6708619021b88b6cc862f752849c337b7e 192.168.10.101:7005
   slots: (0 slots) slave
   replicates da98d89a1eb8f1c436cbe768e066ddf38cd78d28
M: da98d89a1eb8f1c436cbe768e066ddf38cd78d28 192.168.10.101:7001
   slots:[5461-10922] (5462 slots) master
   1 additional replica(s)
S: 5faca3ab4336233c5613b3da0017133315051b52 192.168.10.101:7003
   slots: (0 slots) slave
   replicates c6fbf1a8eead3eca8543ed8a6dc9ea38085edbef
M: c6fbf1a8eead3eca8543ed8a6dc9ea38085edbef 192.168.10.101:7002
   slots:[10923-16383] (5461 slots) master
   1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.

4. 验证

• 验证集群是否创建成功

# ./redis-cli -c -h 192.168.10.101 -p 7000 -a fd43W2%3@-=pi

192.168.10.101:7000> cluster nodes
b6db58366b01af1868be50c23574a1f765a7b3bb 192.168.10.101:7004@17004 slave 83508475cd7d3e783ca680bdd3d5fbae21e7e7cb 0 1643042697042 5 connected
0e49af6708619021b88b6cc862f752849c337b7e 192.168.10.101:7005@17005 slave da98d89a1eb8f1c436cbe768e066ddf38cd78d28 0 1643042696000 6 connected
da98d89a1eb8f1c436cbe768e066ddf38cd78d28 192.168.10.101:7001@17001 master - 0 1643042693980 2 connected 5461-10922
5faca3ab4336233c5613b3da0017133315051b52 192.168.10.101:7003@17003 slave c6fbf1a8eead3eca8543ed8a6dc9ea38085edbef 0 1643042694000 4 connected
83508475cd7d3e783ca680bdd3d5fbae21e7e7cb 192.168.10.101:7000@17000 myself,master - 0 1643042695000 1 connected 0-5460
c6fbf1a8eead3eca8543ed8a6dc9ea38085edbef 192.168.10.101:7002@17002 master - 0 1643042696005 3 connected 10923-16383

• 验证故障转移

kill -9掉7001，发现其从节点7004转变为了master节点。重新启动7001，其变成了slave节点。

# ps -ef |grep redis

# kill -9 7001节点的PID

# ./redis-cli -c -h 192.168.10.101 -p 7000 -a fd43W2%3@-=pi

192.168.10.101:7000> cluster nodes


# nohup ./redis-server  /mnt/e/redis-cluster-new/7001/redis.conf &

结论是主节点故障后，下挂的从节点会升级成主节点，并接替主节点的槽位。旧的主节点上线后，也只能作为其从节点。