Redis的集群(搭建)
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Redis的集群(搭建)相关的知识，希望对你有一定的参考价值。
分布式数据库是把整个数据集按照分区规则映射到多个节点，每个节点负责一部分数据。
Redis Cluster采用虚拟槽分区（引入虚拟槽改进的一致性哈希算法），所有的键根据哈希函数映射到0~16383整数槽内，计算公式：slot=CRC16(key)&16383，找到槽，再找到槽所在的节点。槽是集群内数据管理和迁移的基本单位。
Redis Cluster搭建需要3个步骤
1. 准备节点
Redis Cluster节点数量至少6个才能保证组成完整高可用的集群，每个节点需开启集群模式，配置如下：
# cluster
cluster-enabled yes
cluster-node-timeout 15000
cluster-config-file "nodes-${port}.conf"
cluster-slave-validity-factor 10
cluster-migration-barrier 1
cluster-require-full-coverage yes
启动6个节点，集群模式的Redis除了原有的配置文件redis.conf，又加了一份集群配置文件nodes-${port}.conf，该文件记录了集群内节点信息的变化，如添加/下线节点，故障转移等，由Redis自动维护。
~/6879/data $ cat nodes-6879.conf
51a3c0a30f397cf28d9f36330cb21df1edda25af :0 myself,master - 0 0 0 connected
vars currentEpoch 0 lastVoteEpoch 0
127.0.0.1:6879> info cluster
# Cluster
cluster_enabled:1
127.0.0.1:6879> cluster info
cluster_state:fail
cluster_slots_assigned:0
cluster_slots_ok:0
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:1
cluster_size:0
cluster_current_epoch:0
cluster_my_epoch:0
cluster_stats_messages_sent:0
cluster_stats_messages_received:0
127.0.0.1:6879> cluster nodes
51a3c0a30f397cf28d9f36330cb21df1edda25af :6879 myself,master - 0 0 0 connected
目前每个节点只能识别出自己的信息，并不知道对方的存在，下面通过握手让6个节点彼此建立联系组成一个集群。
2. 节点握手
节点握手是指一批运行在集群模式下的节点通过Gossip协议彼此通信，达到感知对方的过程。在集群内任意节点上执行cluster meet命令加入新节点，握手状态会通过消息在集群内传播，这样其它节点会自动发现新节点并发起握手流程。
127.0.0.1:6879> cluster meet 127.0.0.1 6880
127.0.0.1:6879> cluster meet 127.0.0.1 6881
127.0.0.1:6879> cluster meet 127.0.0.1 6882
127.0.0.1:6879> cluster meet 127.0.0.1 6883
127.0.0.1:6879> cluster meet 127.0.0.1 6884
127.0.0.1:6879> cluster nodes
51a3c0a30f397cf28d9f36330cb21df1edda25af 127.0.0.1:6879 myself,master - 0 0 1 connected
aa94bcbe1f7ebcf850e1d75cad712a0bbc044d97 127.0.0.1:6884 master - 0 1532681718583 5 connected
8c0e31b4cadc12c784eaa63a200fbb9b86e49a72 127.0.0.1:6880 master - 0 1532681713007 2 connected
ae5c6df67610f399491c174a6ee97345e03fd610 127.0.0.1:6882 master - 0 1532681716555 3 connected
b298f797f42b2fe1269f83e9fadd6cbb2af2fd04 127.0.0.1:6881 master - 0 1532681717567 4 connected
ff0092bfdd407ea73f036564e48b277898494ac1 127.0.0.1:6883 master - 0 1532681719596 0 connected
节点握手后还不能正常工作，这时集群处于下线状态，是由于槽没有分配到节点，集群无法完成槽到节点的映射。
127.0.0.1:6879> set hello redis
(error) CLUSTERDOWN Hash slot not served
3. 分配槽
Redis cluster把所有数据映射到16384个槽中，每个key会映射为一个固定的槽，只有当节点分配了槽，才能响应和这些槽关联的键命令，下面通过cluster addslots命令为节点分配槽。
$ redis-cli -p 6879 cluster addslots {0..5641}
$ redis-cli -p 6880 cluster addslots {5642..10922}
$ redis-cli -p 6881 cluster addslots {10923..16383}
当前集群状态是ok，进入在线状态，所有槽都已经分配给节点，cluster nodes命令可看到槽和节点的对应关系。
127.0.0.1:6879> cluster info
cluster_state:ok
cluster_slots_assigned:16384
cluster_slots_ok:16384
cluster_slots_pfail:0
cluster_slots_fail:0
cluster_known_nodes:6
cluster_size:3
cluster_current_epoch:5
cluster_my_epoch:1
cluster_stats_messages_sent:2864
cluster_stats_messages_received:2864
127.0.0.1:6879> cluster nodes
51a3c0a30f397cf28d9f36330cb21df1edda25af 127.0.0.1:6879 myself,master - 0 0 1 connected 0-5641
aa94bcbe1f7ebcf850e1d75cad712a0bbc044d97 127.0.0.1:6884 master - 0 1532683047361 5 connected
8c0e31b4cadc12c784eaa63a200fbb9b86e49a72 127.0.0.1:6880 master - 0 1532683048374 2 connected 5642-10922
ae5c6df67610f399491c174a6ee97345e03fd610 127.0.0.1:6882 master - 0 1532683046350 3 connected
b298f797f42b2fe1269f83e9fadd6cbb2af2fd04 127.0.0.1:6881 master - 0 1532683044329 4 connected 10923-16383
ff0092bfdd407ea73f036564e48b277898494ac1 127.0.0.1:6883 master - 0 1532683045341 0 connected
目前还有3个节点没有使用，作为一个完整的集群，每个负责处理槽的节点应该具有从节点，保证出现故障时，可以进行自动故障转移，使用cluster replicate命令让一个节点成为从节点。
127.0.0.1:6882> cluster replicate 51a3c0a30f397cf28d9f36330cb21df1edda25af
127.0.0.1:6883> cluster replicate 8c0e31b4cadc12c784eaa63a200fbb9b86e49a72
127.0.0.1:6884> cluster replicate b298f797f42b2fe1269f83e9fadd6cbb2af2fd04
127.0.0.1:6884> cluster nodes
8c0e31b4cadc12c784eaa63a200fbb9b86e49a72 127.0.0.1:6880 master - 0 1532683770612 2 connected 5642-10922
ae5c6df67610f399491c174a6ee97345e03fd610 127.0.0.1:6882 slave 51a3c0a30f397cf28d9f36330cb21df1edda25af 0 1532683773646 3 connected
b298f797f42b2fe1269f83e9fadd6cbb2af2fd04 127.0.0.1:6881 master - 0 1532683772637 4 connected 10923-16383
ff0092bfdd407ea73f036564e48b277898494ac1 127.0.0.1:6883 slave 8c0e31b4cadc12c784eaa63a200fbb9b86e49a72 0 1532683774150 2 connected
51a3c0a30f397cf28d9f36330cb21df1edda25af 127.0.0.1:6879 master - 0 1532683769602 1 connected 0-5641
aa94bcbe1f7ebcf850e1d75cad712a0bbc044d97 127.0.0.1:6884 myself,slave b298f797f42b2fe1269f83e9fadd6cbb2af2fd04 0 0 5 connected
目前为止，手动建立了一个6个节点的集群，3个主节点负责处理槽和相关数据，3个从节点负责故障转移。
手动搭建集群，步骤比较繁琐，Redis官方提供了redis-trib.rb工具方便快速搭建，该工具采用Ruby实现，使用需先准备Ruby环境。
准备Ruby环境
# wget https://cache.ruby-lang.org/pub/ruby/2.3/ruby-2.3.1.tar.gz
# tar zxf ruby-2.3.1.tar.gz
# ./configure --prefix=/usr/local/ruby
# make
# make install
# wget http://rubygems.org/downloads/redis-3.3.0.gem
# gem install --local redis-3.3.0.gem
Successfully installed redis-3.3.0
Parsing documentation for redis-3.3.0
Installing ri documentation for redis-3.3.0
Done installing documentation for redis after 0 seconds
1 gem installed
安装Ruby环境后，执行redis-trib.rb命令确认环境是否正确。
$ redis-trib.rb
Usage: redis-trib <command> <options> <arguments ...>
...
创建集群
$ redis-trib.rb create --replicas 1 127.0.0.1:6879 127.0.0.1:6880 127.0.0.1:6881 127.0.0.1:6882 127.0.0.1:6883 127.0.0.1:6884
>>> Creating cluster
>>> Performing hash slots allocation on 6 nodes...
Using 3 masters:
127.0.0.1:6879
127.0.0.1:6880
127.0.0.1:6881
Adding replica 127.0.0.1:6882 to 127.0.0.1:6879
Adding replica 127.0.0.1:6883 to 127.0.0.1:6880
Adding replica 127.0.0.1:6884 to 127.0.0.1:6881
M: 93aec643effa795f33ab2b151c6a2b273eeb5462 127.0.0.1:6879
slots:0-5460 (5461 slots) master
M: 22b6e422aed074e42d295c061f0b4c102304b5bb 127.0.0.1:6880
slots:5461-10922 (5462 slots) master
M: 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b 127.0.0.1:6881
slots:10923-16383 (5461 slots) master
S: 3ae8e6622d37e15701a757139a51c0006a6df664 127.0.0.1:6882
replicates 93aec643effa795f33ab2b151c6a2b273eeb5462
S: 324228642e6fc87a0c367a6b8e7a47e2879aa7d5 127.0.0.1:6883
replicates 22b6e422aed074e42d295c061f0b4c102304b5bb
S: 178525484b9e865522a0cfa2fef7f207413272d3 127.0.0.1:6884
replicates 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b
Can I set the above configuration? (type 'yes' to accept): yes
>>> Nodes configuration updated
>>> Assign a different config epoch to each node
>>> Sending CLUSTER MEET messages to join the cluster
Waiting for the cluster to join...
>>> Performing Cluster Check (using node 127.0.0.1:6879)
M: 93aec643effa795f33ab2b151c6a2b273eeb5462 127.0.0.1:6879
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 22b6e422aed074e42d295c061f0b4c102304b5bb 127.0.0.1:6880
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 3ae8e6622d37e15701a757139a51c0006a6df664 127.0.0.1:6882
slots: (0 slots) slave
replicates 93aec643effa795f33ab2b151c6a2b273eeb5462
M: 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b 127.0.0.1:6881
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 324228642e6fc87a0c367a6b8e7a47e2879aa7d5 127.0.0.1:6883
slots: (0 slots) slave
replicates 22b6e422aed074e42d295c061f0b4c102304b5bb
S: 178525484b9e865522a0cfa2fef7f207413272d3 127.0.0.1:6884
slots: (0 slots) slave
replicates 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
查看集群信息
$ redis-trib.rb info 127.0.0.1:6879
127.0.0.1:6879 (93aec643...) -> 0 keys | 5461 slots | 1 slaves.
127.0.0.1:6880 (22b6e422...) -> 0 keys | 5462 slots | 1 slaves.
127.0.0.1:6881 (1835a6b9...) -> 0 keys | 5461 slots | 1 slaves.
[OK] 0 keys in 3 masters.
0.00 keys per slot on average.
检查集群完整性
redis-trib.rb check 127.0.0.1:6879
>>> Performing Cluster Check (using node 127.0.0.1:6879)
M: 93aec643effa795f33ab2b151c6a2b273eeb5462 127.0.0.1:6879
slots:0-5460 (5461 slots) master
1 additional replica(s)
M: 22b6e422aed074e42d295c061f0b4c102304b5bb 127.0.0.1:6880
slots:5461-10922 (5462 slots) master
1 additional replica(s)
S: 3ae8e6622d37e15701a757139a51c0006a6df664 127.0.0.1:6882
slots: (0 slots) slave
replicates 93aec643effa795f33ab2b151c6a2b273eeb5462
M: 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b 127.0.0.1:6881
slots:10923-16383 (5461 slots) master
1 additional replica(s)
S: 324228642e6fc87a0c367a6b8e7a47e2879aa7d5 127.0.0.1:6883
slots: (0 slots) slave
replicates 22b6e422aed074e42d295c061f0b4c102304b5bb
S: 178525484b9e865522a0cfa2fef7f207413272d3 127.0.0.1:6884
slots: (0 slots) slave
replicates 1835a6b9b3e0c0c08b09a6798c0359996be7ba7b
[OK] All nodes agree about slots configuration.
>>> Check for open slots...
>>> Check slots coverage...
[OK] All 16384 slots covered.
若感兴趣可关注订阅号”数据库最佳实践”(DBBestPractice).
以上是关于Redis的集群(搭建)的主要内容，如果未能解决你的问题，请参考以下文章