Redis cluster命令部署集群及数据导入
Posted njsummer
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Redis cluster命令部署集群及数据导入相关的知识，希望对你有一定的参考价值。
前文实践了Redis原生命令部署集群，redis还提供了cluster命令集，可以简化了集群的部署和管理，本文将利用cluster命令集来完成集群搭建的实践。
官方文档 https://redis.io/docs/manual/scaling/
redis cluster 相关命令可以查看 --cluster 帮助，命令格式：redis-cli --cluster help
1. Redis cluster集群的环境准备
环境准备：
需要准备六台主实现redis集群 及 一台 redis 独立机器完成本实验
时间同步，确保NTP或Chrony服务正常运行。
禁用SELinux和防火墙（或放通需要的端口）
每个redis 节点采用相同的相同的redis版本、相同的密码、硬件配置
所有redis服务器必须没有任何数据
2. Redis cluster 主机基本配置
所有六台主机都执行以下配置，通过SSH终端软件一次性向六台主机发送命令来完成。
## yum 方式安装redis
[root@CentOS84 ]#dnf -y install redis
## 每个节点同时修改redis配置，开启cluster功能参数
[root@CentOS84 ]#ll /etc/redis.conf
-rw-r----- 1 redis root 62189 Oct 20 2021 /etc/redis.conf
[root@CentOS84-IP172-18 ]#sed -i.bak -e s/bind 127.0.0.1/bind 0.0.0.0/ -e /masterauth/a masterauth 123456 -e /# requirepass/a requirepass 123456 -e /# cluster-enabled yes/a cluster-enabled yes -e /# cluster-config-file nodes-6379.conf/a cluster-config-file nodes-6379.conf -e /cluster-require-full-coverage yes/c cluster-require-full-coverage no /etc/redis.conf
## 开机自动启动
[root@CentOS84 ]#systemctl enable --now redis
[root@CentOS84 ]#systemctl restart redis
## 验证当前Redis服务状态
[root@CentOS84 ]#ss -lnt
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0 511 0.0.0.0:16379 0.0.0.0:*
LISTEN 0 511 0.0.0.0:6379 0.0.0.0:*
## 进程标准了[cluster]
[root@CentOS84-IP172-18 ]#ps -ef|grep redis
redis 206879 1 0 16:27 ? 00:00:00 /usr/bin/redis-server 0.0.0.0:6379 [cluster]
root 206929 105260 0 16:29 pts/0 00:00:00 grep --color=auto redis
[root@CentOS84-IP172-18 ]#
3. 创建并验证集群
基本任务：利用redis-cli --cluster-replicas 命令一次性完成六个节点三主三从集群的创建；并通过命令查看验证集群创建情况。
## 通过命令redis-cli --cluster-replicas 1 创建集群，其中1表示每个master对应一个slave节点
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning --cluster create 172.16.0.18:6379 172.16.0.28:6379 172.16.0.38:6379 172.16.0.118:6379 172.16.0.128:6379 172.16.0.138:6379 --cluster-replicas 1
>>> Performing hash slots allocation on 6 nodes...
Master[0] -> Slots 0 - 5460
Master[1] -> Slots 5461 - 10922
Master[2] -> Slots 10923 - 16383
Adding replica 172.16.0.118:6379 to 172.16.0.18:6379
Adding replica 172.16.0.128:6379 to 172.16.0.28:6379
Adding replica 172.16.0.138:6379 to 172.16.0.38:6379
M: c42d1733b3a317becddc484c5a981a5b0ae15566 172.16.0.18:6379 # 带M的为master
slots:[0-5460] (5461 slots) master # 槽位起始和结束位
M: dabcadd4490387830f60d01e361931d010b6c1f2 172.16.0.28:6379
slots:[5461-10922] (5462 slots) master
M: aff40cdf12744b6444f79d7237360d70e70b4b97 172.16.0.38:6379
slots:[10923-16383] (5461 slots) master
S: fc60360518729269b6a105e3684883ee5420a46f 172.16.0.118:6379 # 带S的slave
replicates c42d1733b3a317becddc484c5a981a5b0ae15566
S: f7e1f8a7a65e8fcae1cf47245f9e8c259f501757 172.16.0.128:6379
replicates dabcadd4490387830f60d01e361931d010b6c1f2
S: 86af91ad4a9ee614f240232627700bc7ae17885d 172.16.0.138:6379
replicates aff40cdf12744b6444f79d7237360d70e70b4b97
Can I set the above configuration? (type yes to accept): yes # 输入yes自动创建集群
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join
.....
>>> Performing Cluster Check (using node 172.16.0.18:6379)
M: c42d1733b3a317becddc484c5a981a5b0ae15566 172.16.0.18:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: dabcadd4490387830f60d01e361931d010b6c1f2 172.16.0.28:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: fc60360518729269b6a105e3684883ee5420a46f 172.16.0.118:6379
slots: (0 slots) slave
replicates c42d1733b3a317becddc484c5a981a5b0ae15566
S: 86af91ad4a9ee614f240232627700bc7ae17885d 172.16.0.138:6379
slots: (0 slots) slave
replicates aff40cdf12744b6444f79d7237360d70e70b4b97
S: f7e1f8a7a65e8fcae1cf47245f9e8c259f501757 172.16.0.128:6379
slots: (0 slots) slave
replicates dabcadd4490387830f60d01e361931d010b6c1f2
M: aff40cdf12744b6444f79d7237360d70e70b4b97 172.16.0.38:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration. # 所有节点槽位分配完成
>>> Check for open slots... # 检查打开的槽位
>>> Check slots coverage... # 检查槽位覆盖范围
[OK] All 16384 slots covered. # 所有槽位(16384个)分配完成
[root@CentOS84-IP172-18 ]#
################################################################################
#### 利用redis自带的cluster 工具的一条命令完成了三主三从的redis集群的创建
replica 172.16.0.118:6379 to 172.16.0.18:6379
replica 172.16.0.128:6379 to 172.16.0.28:6379
replica 172.16.0.138:6379 to 172.16.0.38:6379
#### 验证一对主从
[root@CentOS84-IP172-18 ]#redis-cli -h 172.16.0.18 -a 123456 --no-auth-warning info replication
# Replication
role:master
connected_slaves:1
slave0:ip=172.16.0.118,port=6379,state=online,offset=672,lag=1
master_replid:52a9386e8831ceb3b14bf68378b060a7336b9c13
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:672
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:672
[root@CentOS84-IP172-18 ]#redis-cli -h 172.16.0.118 -a 123456 --no-auth-warning info replication
# Replication
role:slave
master_host:172.16.0.18
master_port:6379
master_link_status:up
master_last_io_seconds_ago:1
master_sync_in_progress:0
slave_repl_offset:686
slave_priority:100
slave_read_only:1
connected_slaves:0
master_replid:52a9386e8831ceb3b14bf68378b060a7336b9c13
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:686
second_repl_offset:-1
repl_backlog_active:1
repl_backlog_size:1048576
repl_backlog_first_byte_offset:1
repl_backlog_histlen:686
[root@CentOS84-IP172-18 ]#
################################################################################
#### 验证集群状态
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:6
cluster_my_epoch:1
cluster_stats_messages_ping_sent:575
cluster_stats_messages_pong_sent:583
cluster_stats_messages_sent:1158
cluster_stats_messages_ping_received:578
cluster_stats_messages_pong_received:575
cluster_stats_messages_meet_received:5
cluster_stats_messages_received:1158
## 查看任意节点的集群状态
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning --cluster info 172.16.0.18:6379
172.16.0.18:6379 (c42d1733...) -> 0 keys | 5461 slots | 1 slaves.
172.16.0.28:6379 (dabcadd4...) -> 0 keys | 5462 slots | 1 slaves.
172.16.0.38:6379 (aff40cdf...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
[root@CentOS84-IP172-18 ]#
################################################################################
#### 查看集群node对应关系
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning cluster nodes
dabcadd4490387830f60d01e361931d010b6c1f2 172.16.0.28:6379@16379 master - 0 1651826734642 2 connected 5461-10922
c42d1733b3a317becddc484c5a981a5b0ae15566 172.16.0.18:6379@16379 myself,master - 0 1651826731000 1 connected 0-5460
fc60360518729269b6a105e3684883ee5420a46f 172.16.0.118:6379@16379 slave c42d1733b3a317becddc484c5a981a5b0ae15566 0 1651826732639 4 connected
86af91ad4a9ee614f240232627700bc7ae17885d 172.16.0.138:6379@16379 slave aff40cdf12744b6444f79d7237360d70e70b4b97 0 1651826733000 6 connected
f7e1f8a7a65e8fcae1cf47245f9e8c259f501757 172.16.0.128:6379@16379 slave dabcadd4490387830f60d01e361931d010b6c1f2 0 1651826732000 5 connected
aff40cdf12744b6444f79d7237360d70e70b4b97 172.16.0.38:6379@16379 master - 0 1651826734000 3 connected 10923-16383
[root@CentOS84-IP172-18 ]#
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 --no-auth-warning --cluster check 172.16.0.18:6379
172.16.0.18:6379 (c42d1733...) -> 0 keys | 5461 slots | 1 slaves.
172.16.0.28:6379 (dabcadd4...) -> 0 keys | 5462 slots | 1 slaves.
172.16.0.38:6379 (aff40cdf...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
>>> Performing Cluster Check (using node 172.16.0.18:6379)
M: c42d1733b3a317becddc484c5a981a5b0ae15566 172.16.0.18:6379
slots:[0-5460] (5461 slots) master
1 additional replica(s)
M: dabcadd4490387830f60d01e361931d010b6c1f2 172.16.0.28:6379
slots:[5461-10922] (5462 slots) master
1 additional replica(s)
S: fc60360518729269b6a105e3684883ee5420a46f 172.16.0.118:6379
slots: (0 slots) slave
replicates c42d1733b3a317becddc484c5a981a5b0ae15566
S: 86af91ad4a9ee614f240232627700bc7ae17885d 172.16.0.138:6379
slots: (0 slots) slave
replicates aff40cdf12744b6444f79d7237360d70e70b4b97
S: f7e1f8a7a65e8fcae1cf47245f9e8c259f501757 172.16.0.128:6379
slots: (0 slots) slave
replicates dabcadd4490387830f60d01e361931d010b6c1f2
M: aff40cdf12744b6444f79d7237360d70e70b4b97 172.16.0.38:6379
slots:[10923-16383] (5461 slots) master
1 additional replica(s)
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
[root@CentOS84-IP172-18 ]#
4. 手工命令验证集群写入数据
#### 验证集群写入key
################################################################################
# 集群会经过算法计算，当前key的槽位需要写入指定的node
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 -h 172.16.0.18 --no-auth-warning set key1 shone001
(error) MOVED 9189 172.16.0.28:6379
# 槽位不在当前node所以无法写入
# 指定槽位对应node可写入
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 -h 172.16.0.28 --no-auth-warning set key1 shone001
OK
[root@CentOS84-IP172-18 ]#
# 随机查询一个key，集群会自动按照算法得出应该在哪个节点的槽位内找对应数据
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 -h 172.16.0.18 --no-auth-warning get key1
(error) MOVED 9189 172.16.0.28:6379 # 告知在IP28的槽位内
# 在相应槽位上去才能查到对应的值
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 -h 172.16.0.28 --no-auth-warning get key1
"shone001"
# 在其自身的slave节点上可以用 keys "*" 看到键，但是并不支持 get 去查询值
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 -h 172.16.0.128 --no-auth-warning keys "*"
1) "key1"
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 -h 172.16.0.128 --no-auth-warning get key1
(error) MOVED 9189 172.16.0.28:6379
[root@CentOS84-IP172-18 ]#
################################################################################
#### redis集群计算得到shone和key1 对应的slot 值
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 -h 172.16.0.18 --no-auth-warning cluster keyslot shone
(integer) 3129
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 -h 172.16.0.18 --no-auth-warning cluster keyslot key1
(integer) 9189
[root@CentOS84-IP172-18 ]#
################################################################################
# 使用选项-c 以集群模式连接
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 -c --no-auth-warning
127.0.0.1:6379> cluster keyslot shone
(integer) 3129
127.0.0.1:6379> set shone LTD_XJX
OK
127.0.0.1:6379> get shone
"LTD_XJX"
127.0.0.1:6379> exit
[root@CentOS84-IP172-18 ]#redis-cli -a 123456 -c --no-auth-warning get shone
"LTD_XJX"
[root@CentOS84-IP172-18 ]#
5. 利用python程序实现RedisCluster集群写入
官网：https://github.com/Grokzen/redis-py-cluster
#### 本次是在 IP08上去安装并准备脚本对集群进行远程数据写入，容易理解错误哈。
################################################################################
# 安装redis-cli 的工具
[root@CentOS84-IP172-08 ]#yum install -y redis<以上是关于Redis cluster命令部署集群及数据导入的主要内容，如果未能解决你的问题，请参考以下文章