Redis HA+Sentinel

Posted 2020-08-20

环境：Centos 6.6

redis version：3.2.5

sentinel是一个管理redis实例的工具，它可以实现对redis的监控、通知、自动故障转移。sentinel不断的检测redis实例是否可以正常工作，通过API向其他程序报告redis的状态，如果redis master不能工作，则会自动启动故障转移进程，将其中的一个slave提升为master，其他的slave重新设置新的master服务器。

配置redis HA+哨兵(sentinel)：

在redis1上配置：

[[email protected] ~]# cd /usr/local/redis-3.2.5/
[[email protected] redis-3.2.5]# make
[[email protected] redis-3.2.5]# make PREFIX=/usr/local/redis-sentinel install
[[email protected] redis-3.2.5]# make test

 

[[email protected] etc]# cd /usr/local/redis-3.2.5/
[[email protected] redis-3.2.5]# cp redis.conf redis.conf.default
[[email protected] ~]# mkdir -p /usr/local/redis-3.2.5/{data1,data2}

redis1 （6379）配置：
[[email protected] ~]# grep -v ‘^#‘ /usr/local/redis-3.2.5/redis_6379.conf | grep -v ‘^$‘  
bind 192.168.31.100
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile "/var/run/redis_6379.pid"
loglevel notice
logfile "/var/log/redis_6379.log"
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename "dump_6379.rdb"
dir "/usr/local/redis-3.2.5/data1"
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
requirepass 1234567
maxmemory-policy volatile-lru
maxmemory 8gb
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite yes
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
[[email protected] ~]#

redis2（6380）配置：
[[email protected] ~]# grep -v ‘^#‘ /usr/local/redis-3.2.5/redis_6380.conf | grep -v ‘^$‘  
bind 192.168.31.100
protected-mode yes
port 6380
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile "/var/run/redis_6380.pid"
loglevel notice
logfile "/var/log/redis_6380.log"
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename "dump_6380.rdb"
dir "/usr/local/redis-3.2.5/data2"
masterauth "1234567"
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
maxmemory-policy volatile-lru
maxmemory 8gb
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
[[email protected] ~]#

[[email protected] redis-3.2.5]# cd /usr/local/redis-sentinel/
[[email protected] redis-sentinel]# mkdir etc log data
[[email protected] redis-sentinel]# cp /usr/local/redis-3.2.5/sentinel.conf /usr/local/redis-sentinel/etc/
[[email protected] redis-sentinel]# cd etc/

 

redis Sentinel1的配置：

[[email protected] ~]# grep -v ‘^#‘ /usr/local/redis-sentinel/etc/sentinel.conf | grep -v ‘^$‘ 
bind 192.168.31.100
port 26379
daemonize yes
dir "/usr/local/redis-sentinel/data"
logfile "/usr/local/redis-sentinel/log/redis-sentinel.log"
pidfile "/usr/local/redis-sentinel/sentinel.pid"
sentinel monitor mymaster 192.168.31.100 6379 1
sentinel down-after-milliseconds mymaster 15000

[[email protected] ~]# ln -s /usr/local/redis-3.2.5/src/redis-server /bin/
[[email protected] ~]# ln -s /usr/local/redis-3.2.5/src/redis-cli /bin/
[[email protected] ~]# ln -s /usr/local/redis-sentinel/bin/redis-sentinel /bin/

在redis3配置：

[[email protected] ~]# cd /usr/local/redis-3.2.5/
[[email protected] redis-3.2.5]# make
[[email protected] redis-3.2.5]# make PREFIX=/usr/local/redis-sentinel install
[[email protected] redis-3.2.5]# make test
[[email protected] etc]# cd /usr/local/redis-3.2.5/
[[email protected] redis-3.2.5]# cp redis.conf redis.conf.default
[[email protected] ~]# mkdir -p /usr/local/redis-3.2.5/data

redis3（6379）的配置：
[[email protected] ~]# grep -v ‘^#‘ /usr/local/redis-3.2.5/redis.conf | grep -v ‘^$‘  
bind 192.168.31.110
protected-mode yes
port 6379
tcp-backlog 511
timeout 0
tcp-keepalive 300
daemonize yes
supervised no
pidfile "/var/run/redis_6379.pid"
loglevel notice
logfile "/var/log/redis_6379.log"
databases 16
save 900 1
save 300 10
save 60 10000
stop-writes-on-bgsave-error yes
rdbcompression yes
rdbchecksum yes
dbfilename "dump_6379.rdb"
dir "/usr/local/redis-3.2.5/data"
slaveof 192.168.31.100 6380
masterauth "1234567"
slave-serve-stale-data yes
slave-read-only yes
repl-diskless-sync no
repl-diskless-sync-delay 5
repl-disable-tcp-nodelay no
slave-priority 100
maxmemory-policy volatile-lru
maxmemory 8gb
appendonly yes
appendfilename "appendonly.aof"
appendfsync everysec
no-appendfsync-on-rewrite no
auto-aof-rewrite-percentage 100
auto-aof-rewrite-min-size 64mb
aof-load-truncated yes
lua-time-limit 5000
slowlog-log-slower-than 10000
slowlog-max-len 128
latency-monitor-threshold 0
notify-keyspace-events ""
hash-max-ziplist-entries 512
hash-max-ziplist-value 64
list-max-ziplist-size -2
list-compress-depth 0
set-max-intset-entries 512
zset-max-ziplist-entries 128
zset-max-ziplist-value 64
hll-sparse-max-bytes 3000
activerehashing yes
client-output-buffer-limit normal 0 0 0
client-output-buffer-limit slave 256mb 64mb 60
client-output-buffer-limit pubsub 32mb 8mb 60
hz 10
aof-rewrite-incremental-fsync yes
[[email protected] ~]#

[[email protected] redis-3.2.5]# cd /usr/local/redis-sentinel/
[[email protected] redis-sentinel]# mkdir etc log data
[[email protected] redis-sentinel]# cp /usr/local/redis-3.2.5/sentinel.conf /usr/local/redis-sentinel/etc/
[[email protected] redis-sentinel]# cd etc/




redis sentinel2配置：

[[email protected] ~]# grep -v ‘^#‘ /usr/local/redis-sentinel/etc/sentinel.conf | grep -v ‘^$‘
bind 192.168.31.110
port 26379
daemonize yes
dir "/usr/local/redis-sentinel/data"
logfile "/usr/local/redis-sentinel/log/redis-sentinel.log"
pidfile "/usr/local/redis-sentinel/sentinel.pid"
sentinel monitor mymaster 192.168.31.100 6379 1
sentinel down-after-milliseconds mymaster 15000
sentinel failover-timeout mymaster 10000
sentinel auth-pass mymaster 1234567     //需要和redis验证的密码一致
sentinel config-epoch mymaster 15
sentinel leader-epoch mymaster 8397
sentinel current-epoch 8397
[[email protected] ~]#

[[email protected] ~]# ln -s /usr/local/redis-3.2.5/src/redis-server /bin/
[[email protected] ~]# ln -s /usr/local/redis-3.2.5/src/redis-cli /bin/
[[email protected] ~]# ln -s /usr/local/redis-sentinel/bin/redis-sentinel /bin/

注：要先启动所以的sentinel再逐一启动各个redis：

[[email protected] ~]# redis-sentinel /usr/local/redis-sentinel/etc/sentinel.conf
[[email protected] ~]# redis-sentinel /usr/local/redis-sentinel/etc/sentinel.conf
[[email protected] ~]# redis-server /usr/local/redis-3.2.5/redis_6379.conf
[[email protected] ~]# redis-server /usr/local/redis-3.2.5/redis_6380.conf
[[email protected] ~]# redis-server /usr/local/redis-3.2.5/redis.conf
[[email protected] ~]# echo "redis-sentinel /usr/local/redis-sentinel/etc/sentinel.conf" >> /etc/rc.local
[[email protected] ~]# echo "redis-server /usr/local/redis-3.2.5/redis_6379.conf" >> /etc/rc.local
[[email protected] ~]# echo "redis-server /usr/local/redis-3.2.5/redis_6380.conf" >> /etc/rc.local
[[email protected] ~]# echo "redis-sentinel /usr/local/redis-sentinel/etc/sentinel.conf" >> /etc/rc.local
[[email protected] ~]# echo "redis-server /usr/local/redis-3.2.5/redis.conf" >> /etc/rc.local

 

 

日志报错：

解决办法：

[[email protected] ~]# tail -4 /etc/sysctl.conf
#redis
net.core.somaxconn = 20480
net.ipv4.tcp_max_syn_backlog = 20480
vm.overcommit_memory =1
[[email protected] ~]# sysctl -p
[[email protected] ~]# echo never > /sys/kernel/mm/transparent_hugepage/enabled
[[email protected] ~]# echo "echo never > /sys/kernel/mm/transparent_hugepage/enabled" >> /etc/rc.local

 

主节点测试写入数据：

 

在redis slave上查看数据同步情况，slave是不能写入数据的：

 

成功。

 

测试redis Master宕机后，哨兵是否能正常切换：

[[email protected] ~]# /usr/local/redis-sentinel/bin/redis-cli -h 127.0.0.1 -p 26379
127.0.0.1:26379> info

 

 

 

 

 

OK，切换成功，之前的Master变成slave，其中一个slave变成Master

 

Sentinel日志：

重新启动之前宕机的redis1，日志显示其启动切换角色为slave

本文出自 “鹏城-酱油瓶” 博客，谢绝转载！