PostgreSQL 通过归档日志定期增量同步数据

Posted 2020-12-15 狂神314

通过归档日志定期增量同步数据实验：

一、再次实验从归档持续同步的方式：
1.修改主库，开启归档，将日志归档到从库：
archive_mode = on
archive_command = ‘ssh 192.168.5.133 test ! -f /home/kingbase/test_fy/archive/%f && scp %p 192.168.5.133:/home/kingbase/test_fy/archive/%f‘
wal_keep_segments = 1024
max_wal_senders = 8

2.备库需要从主库拉取全量备份：
pg_basebackup -D ./data -R -h 192.168.5.132 -p 54328 -U SYSTEM -w

3.备库recovery.conf文件：
restore_command = ‘cp /home/kingbase/test_fy/archive/%f %p‘
standby_mode = on

这种方法是同步的，当备库需要的日志没有时，会一直提示：
LOG: restored log file "00000001000000000000004E" from archive
cp: cannot stat `/home/kingbase/test_fy/archive/00000001000000000000004F‘: No such file or directory
cp: cannot stat `/home/kingbase/test_fy/archive/00000001000000000000004F‘: No such file or directory
cp: cannot stat `/home/kingbase/test_fy/archive/00000001000000000000004F‘: No such file or directory
cp: cannot stat `/home/kingbase/test_fy/archive/00000001000000000000004F‘: No such file or directory

二、在一的基础上，定期拷贝wal文件
1.关闭归档，并重启数据库：
2.产生新的数据，并拷贝到从库归档路径：
scp /home/kingbase/test_fy/data/sys_xlog/* 192.168.5.133:/home/kingbase/test_fy/archive/

3.从库数据库会从归档目录中获取到增量的日志并回放
cp: cannot stat `/home/kingbase/test_fy/archive/000000010000000000000050‘: No such file or directory
cp: cannot stat `/home/kingbase/test_fy/archive/000000010000000000000050‘: No such file or directory
cp: cannot stat `/home/kingbase/test_fy/archive/000000010000000000000050‘: No such file or directory
cp: cannot stat `/home/kingbase/test_fy/archive/000000010000000000000050‘: No such file or directory
cp: cannot stat `/home/kingbase/test_fy/archive/000000010000000000000050‘: No such file or directory
LOG: restored log file "000000010000000000000050" from archive
LOG: restored log file "000000010000000000000051" from archive
LOG: restored log file "000000010000000000000052" from archive

三、结论：
通过日志文件拷贝的方式，可以实现增量数据的交换和同步

备注：
1）wal日志拷贝可以从主库归档目录中获取。
2）拷贝完成的wal日志，可以从归档中删除。例如，写一个脚本，同步前一天的数据，并删除。
3）备库始终是只读的。