redis集群同步迁移方法：通过redis-migrate-tool实现

Posted 2020-08-16 天午绝人的博客

前篇介绍的redis replication方法，操作步骤多，而且容易出错。在git上看到一些开源工具也能实现同步迁移功能，而且步骤简单，比如redis-port,redis-migrate-tool等工具。实验演示使用redis-migrate-tool，将redis cluster 迁移到一个单实例redis中。

1.redis-migrate-tool的安装

见https://github.com/vipshop/redis-migrate-tool

需要注意的是安装redis-migrate-tool依赖automake, libtool, autoconf and bzip2这些包

2..运行环境

源节点：10.86.30.37:36379/10.86.30.37:36380/10.86.30.37:36381

[[email protected]_86_30_37_10.86.30.37 redis-migrate-tool]# redis-cli -c -p 36379 -h 10.86.30.37 cluster nodes

7fab85269fe72d68414ffd15a54605d45f280aff 10.86.30.37:36381 master - 0 1477555329981 7 connected 10001-16383

e0c45fe484e55967c968814076b0a5f67f4f6821 10.86.30.37:36380 master - 0 1477555330984 8 connected 5001-10000

ba3d6a50ef6bdf6212c0360baec97f29f3b25385 10.86.30.37:36379 myself,master - 0 0 6 connected 0-5000

目标节点：10.86.30.37:6389

3.迁移过程

• redis cluster向redis单实例迁移配置文件

[source]
type: redis cluster
servers:
 - 10.86.30.37:36379
 
[target]
type: single
servers:
 - 10.86.30.37:6389
 
[common]
listen: 0.0.0.0:8888

• 执行命令进行同步

/usr/local/redis-migrate-tool/src/redis-migrate-tool -c mgr.conf -d

• 查看同步后的文件

[[email protected]_86_30_37_10.86.30.37 redis]# redis-cli -h 10.86.30.37 -p 6389 keys \*
1) "5"
2) "2"
3) "1"
4) "10"
5) "12"
6) "3"
7) "4"
8) "13"
9) "11"
[[email protected]_86_30_37_10.86.30.37 redis]# redis-cli -h 10.86.30.37 -p 36379 keys \*
1) "10"
2) "11"
3) "3"
[[email protected]_86_30_37_10.86.30.37 redis]# redis-cli -h 10.86.30.37 -p 36380 keys \*
1) "12"
2) "1"
3) "2"
4) "5"
[[email protected]_86_30_37_10.86.30.37 redis]# redis-cli -h 10.86.30.37 -p 36381 keys \*
1) "13"
2) "4"

所有节点数据同步到了但实例节点中，插入数据测试

[[email protected]_86_30_37_10.86.30.37 redis]# redis-cli -h 10.86.30.37 -p 36379 -c
10.86.30.37:36379> set 30 30
-> Redirected to slot [9877] located at 10.86.30.37:36380
OK
10.86.30.37:36380> set 31 31
-> Redirected to slot [14004] located at 10.86.30.37:36381
OK
10.86.30.37:36381> set 32 32
-> Redirected to slot [1751] located at 10.86.30.37:36379
OK 
[[email protected]_86_30_37_10.86.30.37 redis]# redis-cli -h 10.86.30.37 -p 6389 keys \*
 1) "5"
 2) "30"
 3) "31"
 4) "2"
 5) "1"
 6) "10"
 7) "21"
 8) "12"
 9) "3"
10) "4"
11) "13"
12) "32"
13) "11"

可以看到新增数据也同步到了redis单实例节点中

4.利用redis-migrate-tool的工具进行更全面的验证

• 利用redis-migrate-tool提供的命令进行一致性校验：

[[email protected]0_86_30_37_10.86.30.37 redis]# /usr/local/redis-migrate-tool/src/redis-migrate-tool -c mgr.conf log -C redis_check
Check job is running...
 
Checked keys: 1000
Inconsistent value keys: 0
Inconsistent expire keys : 0
Other check error keys: 0
Checked OK keys: 1000
 
All keys checked OK!
Check job finished, used 0.047s

• 利用redis-migrate-tool提供的命令进行插入校验：

[[email protected]_86_30_37_10.86.30.37 redis]# /usr/local/redis-migrate-tool/src/redis-migrate-tool -c mgr.conf log -C redis_testinsert
Test insert job is running...
 
Insert string keys: 200
Insert list keys  : 200
Insert set keys   : 200
Insert zset keys  : 200
Insert hash keys  : 200
Insert total keys : 1000
 
Correct inserted keys: 1000
Test insert job finished, used 0.417s

需要注意的是，插入校验生成的数据并不会清除，如果只是为了测试功能，那么可以减少插入的key。