redis-full-check校验2个不同redis实例数据

Posted 2021-03-11

redis-full-check校验redis数据是否一致：

校验2个不同的redis实例数据：

6986 为redis实例一
6987 为redis实例二

登录6986 redis实例一，模拟设置4个key值

[root@localhost redis-full-check-1.4.7]# redis-cli -h 127.0.0.1 -p 6986 -a ‘Y2hJKSGtuEq‘
Warning: Using a password with ‘-a‘ option on the command line interface may not be safe.
127.0.0.1:6986> keys *
(empty list or set)
127.0.0.1:6986> set test001 001
OK
127.0.0.1:6986> set test002 002
OK
127.0.0.1:6986> set test003 003
OK
127.0.0.1:6986> set test004 004
OK
127.0.0.1:6986> keys *
1) "test003"
2) "test002"
3) "test004"
4) "test001"
127.0.0.1:6986>

登录6987 redis实例二，模拟设置2个key值

[root@localhost redis-full-check-1.4.7]# redis-cli -h 127.0.0.1 -p 6987 -a ‘Y2hJKSGtuEq‘
Warning: Using a password with ‘-a‘ option on the command line interface may not be safe.
127.0.0.1:6987> keys *
(empty list or set)
127.0.0.1:6987> set test002 a02
OK
127.0.0.1:6987> set test004 a04
OK
127.0.0.1:6987> keys *
1) "test004"
2) "test002"
127.0.0.1:6987> 

-m参数指定校验的模式，校验的模式分为4种：
-m参数 比较模式，1表示全量比较，2表示只对比value的长度，3只对比key是否存在，4全量比较的情况下，忽略大key的比较

从key名称来看，6987实例中的key是6986实例key的子集：
不指定校验模式的话，默认采用的是-m 3 只对比key是否存在。建议在校验不同redis实例数据时，采用-m 1全量比较模式

把6987实例作为原实例，6986实例作为目标实例，进行校验:

[root@localhost redis-full-check-1.4.7]# redis-full-check   -s 127.0.0.1:6987  -p ‘Y2hJKSGtuEq‘ -t 127.0.0.1:6986 -a ‘Y2hJKSGtuEq‘  --log=log  --result=result
[root@localhost redis-full-check-1.4.7]# ls
log  redis-full-check  result  result.db.1  result.db.2  result.db.3

[root@localhost redis-full-check-1.4.7]# sqlite3 result.db.3 
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from key;
sqlite> select * from field;
sqlite>

从key名称来看，由于6987实例中的key是6986实例key的子集，所以不存在差异

同时从日志中也可以看到:

[INFO 2020-02-08-12:29:32 main.go:65]: init log success
[INFO 2020-02-08-12:29:32 main.go:164]: configuration: {127.0.0.1:6987 Y2hJKSGtuEq auth 0 -1 127.0.0.1:6986 Y2hJKSGtuEq auth 0 -1 result.db result 3 2 unknown unknown unknown 15000 5 256 5 lo
g  false 16384  20445 false}
[INFO 2020-02-08-12:29:32 main.go:166]: ---------
[INFO 2020-02-08-12:29:32 full_check.go:238]: sourceDbType=0, p.sourcePhysicalDBList=[meaningless]
[INFO 2020-02-08-12:29:32 full_check.go:243]: db=0:keys=2
[INFO 2020-02-08-12:29:32 full_check.go:253]: ---------------- start 1th time compare
[INFO 2020-02-08-12:29:32 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:29:32 scan.go:20]: build connection[source redis addr: [127.0.0.1:6987]]
[INFO 2020-02-08-12:29:33 full_check.go:203]: stat:
times:1, db:0, dbkeys:2, finish:33%, finished:true
KeyScan:{2 2 0}

[INFO 2020-02-08-12:29:33 full_check.go:250]: wait 5 seconds before start
[INFO 2020-02-08-12:29:38 full_check.go:253]: ---------------- start 2th time compare
[INFO 2020-02-08-12:29:38 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:29:38 full_check.go:203]: stat:
times:2, db:0, finished:true
KeyScan:{0 0 0}

[INFO 2020-02-08-12:29:38 full_check.go:250]: wait 5 seconds before start
[INFO 2020-02-08-12:29:43 full_check.go:253]: ---------------- start 3th time compare
[INFO 2020-02-08-12:29:43 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:29:43 full_check.go:203]: stat:
times:3, db:0, finished:true
KeyScan:{0 0 0}

[INFO 2020-02-08-12:29:43 full_check.go:328]: --------------- finished! ----------------
all finish successfully, totally 0 key(s) and 0 field(s) conflict

但是把原实例和目标实例互换下比对，就找到不一致了

把6986实例作为原实例，6987实例作为目标实例，进行校验：

redis-full-check   -s 127.0.0.1:6986  -p ‘Y2hJKSGtuEq‘ -t 127.0.0.1:6987 -a ‘Y2hJKSGtuEq‘  --log=log  --result=result
[root@localhost redis-full-check-1.4.7]# sqlite3 result.db.3 
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from field;
sqlite> select * from key;
1|test001|string|lack_target|0|3|0
2|test003|string|lack_target|0|3|0

查看到6986实例比6987实例多出了2个key

从日志中也可以看到不同：

[INFO 2020-02-08-12:35:10 main.go:65]: init log success
[INFO 2020-02-08-12:35:10 main.go:164]: configuration: {127.0.0.1:6986 Y2hJKSGtuEq auth 0 -1 127.0.0.1:6987 Y2hJKSGtuEq auth 0 -1 result.db result 3 2 unknown unknown unknown 15000 5 256 5 lo
g  false 16384  20445 false}
[INFO 2020-02-08-12:35:10 main.go:166]: ---------
[INFO 2020-02-08-12:35:10 full_check.go:238]: sourceDbType=0, p.sourcePhysicalDBList=[meaningless]
[INFO 2020-02-08-12:35:10 full_check.go:243]: db=0:keys=4
[INFO 2020-02-08-12:35:10 full_check.go:253]: ---------------- start 1th time compare
[INFO 2020-02-08-12:35:10 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:35:10 scan.go:20]: build connection[source redis addr: [127.0.0.1:6986]]
[INFO 2020-02-08-12:35:11 full_check.go:203]: stat:
times:1, db:0, dbkeys:4, finish:33%, finished:true
KeyScan:{4 4 0}
KeyConflictInProcess|string|lack_target|{2 2 0}

[INFO 2020-02-08-12:35:11 full_check.go:250]: wait 5 seconds before start
[INFO 2020-02-08-12:35:16 full_check.go:253]: ---------------- start 2th time compare
[INFO 2020-02-08-12:35:16 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:35:17 full_check.go:203]: stat:
times:2, db:0, finished:true
KeyScan:{2 2 0}
KeyConflictInProcess|string|lack_target|{2 2 0}

[INFO 2020-02-08-12:35:17 full_check.go:250]: wait 5 seconds before start
[INFO 2020-02-08-12:35:22 full_check.go:253]: ---------------- start 3th time compare
[INFO 2020-02-08-12:35:22 full_check.go:278]: start compare db 0
[INFO 2020-02-08-12:35:23 full_check.go:203]: stat:
times:3, db:0, finished:true
KeyScan:{2 2 0}
KeyConflictAtLast|string|lack_target|{2 2 0}
[INFO 2020-02-08-12:35:23 full_check.go:328]: --------------- finished! ----------------
all finish successfully, totally 4 key(s) and 0 field(s) conflict

value: field存在于源端key和目的端key，但是field对应的value不同
登录6987redis实例，修改key test004 为a04; 修改key test003 a03

[root@localhost redis-full-check-1.4.7]# redis-cli -h 127.0.0.1 -p 6987 -a ‘Y2hJKSGtuEq‘
Warning: Using a password with ‘-a‘ option on the command line interface may not be safe.
127.0.0.1:6987> get test004
"a04"
127.0.0.1:6987> get test002
"002"
127.0.0.1:6987> get test001
"001"
127.0.0.1:6987> get test003
"a03"

此时采用redis-full_check全量校验模式：

[root@localhost redis-full-check-1.4.7]# redis-full-check   -s 127.0.0.1:6987  -p ‘Y2hJKSGtuEq‘ -t 127.0.0.1:6986 -a ‘Y2hJKSGtuEq‘  --log=log  --result=result -m 1
[root@localhost redis-full-check-1.4.7]# redis-full-check   -s 127.0.0.1:6987  -p ‘Y2hJKSGtuEq‘ -t 127.0.0.1:6986 -a ‘Y2hJKSGtuEq‘  --log=log  --result=result -m 4

[root@localhost redis-full-check-1.4.7]#  redis-full-check   -s 127.0.0.1:6986  -p ‘Y2hJKSGtuEq‘ -t 127.0.0.1:6987 -a ‘Y2hJKSGtuEq‘    --result=result -m 1
[INFO 2020-02-08-14:44:36 main.go:65]: init log success
........
........
[INFO 2020-02-08-14:44:49 full_check.go:328]: --------------- finished! ----------------
all finish successfully, totally 4 key(s) and 0 field(s) conflict

日志中看到2实例存在数据不一致

查看到实例6986 和实例6987 中key test003 和test004 的value值不一致:

[root@localhost redis-full-check-1.4.7]# sqlite3 result.db.3 
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from key;
1|test004|string|value|0|3|3
2|test003|string|value|0|3|3
sqlite> 

登录6987redis实例，删除key test001:

[root@localhost redis-full-check-1.4.7]# redis-cli -h 127.0.0.1 -p 6987 -a ‘Y2hJKSGtuEq‘
Warning: Using a password with ‘-a‘ option on the command line interface may not be safe.
127.0.0.1:6987> get test001
"001"
127.0.0.1:6987> get test002
"002"
127.0.0.1:6987> get test003
"a03"
127.0.0.1:6987> get test004
"a04"
127.0.0.1:6987> del test002
(integer) 1
127.0.0.1:6987> keys *
1) "test001"
2) "test004"
3) "test003"

此时采用redis-full_check全量校验模式：

[root@localhost redis-full-check-1.4.7]#  redis-full-check   -s 127.0.0.1:6986  -p ‘Y2hJKSGtuEq‘ -t 127.0.0.1:6987 -a ‘Y2hJKSGtuEq‘    --result=result -m 1
[INFO 2020-02-08-14:54:25 main.go:65]: init log success
...............
...............
[INFO 2020-02-08-14:54:38 full_check.go:328]: --------------- finished! ----------------
all finish successfully, totally 6 key(s) and 0 field(s) conflict

发现实例6986 和实例6987中存在key和value值不一致:

[root@localhost redis-full-check-1.4.7]# sqlite3 result.db.3 
SQLite version 3.7.17 2013-05-20 00:56:22
Enter ".help" for instructions
Enter SQL statements terminated with a ";"
sqlite> select * from key;
1|test004|string|value|0|3|3
2|test003|string|value|0|3|3
3|test002|string|lack_target|0|3|0
sqlite> select * from field;