pt-table-checksum
Posted P10ZHUO
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了pt-table-checksum相关的知识，希望对你有一定的参考价值。
pt-table-checksum
pt-table-checksum是社区开源产品。但是它不是mysql自带的工具，需要我们自己进行安装。下面就说下安装过程
安装
版本信息：xtrabackup version 8.0.26-18 based on MySQL server 8.0.26 Linux (x86_64) (revision id: 4aecf82)
MySQL版本：Server version: 8.0.27
os：Red Hat Enterprise Linux Server release 7.9 (Maipo)
下载安装包
version:选择最新的
software：选择rpm，就选RHELx。如果是二进制，就选 Linux-Generic.
这里我们下载rpm包，选择rpm安装。
rpm安装
直接rpm安装，会有如下报错：
[root@test28 local]# rpm -ivh percona-toolkit-3.3.1-1.el7.x86_64.rpm
warning: percona-toolkit-3.3.1-1.el7.x86_64.rpm: Header V4 RSA/SHA256 Signature, key ID 8507efa5: NOKEY
error: Failed dependencies:
perl(IO::Socket::SSL) is needed by percona-toolkit-3.3.1-1.el7.x86_64
显示有包依赖关系，为了避免包依赖，我们选择yum localinstall来安装，自动安装依赖包。
配置本地yum源：
[root@test28 local]# cat /etc/yum.repos.d/base.repo
[base]
name=base
baseurl=file:///mnt/
gpgcheck=0
enabled=1
挂载镜像：
[root@test28 local]# mount /dev/sr0 /mnt/
安装：
[root@test28 local]# yum localinstall percona-toolkit-3.3.1-1.el7.x86_64.rpm
Loaded plugins: product-id, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
Examining percona-toolkit-3.3.1-1.el7.x86_64.rpm: percona-toolkit-3.3.1-1.el7.x86_64
Marking percona-toolkit-3.3.1-1.el7.x86_64.rpm to be installed
Resolving Dependencies
--> Running transaction check
---> Package percona-toolkit.x86_64 0:3.3.1-1.el7 will be installed
--> Processing Dependency: perl(IO::Socket::SSL) for package: percona-toolkit-3.3.1-1.el7.x86_64
c6-media | 2.8 kB 00:00:00
--> Running transaction check
---> Package perl-IO-Socket-SSL.noarch 0:1.94-7.el7 will be installed
--> Processing Dependency: perl-Net-SSLeay >= 1.55-5 for package: perl-IO-Socket-SSL-1.94-7.el7.noarch
--> Processing Dependency: perl(Net::SSLeay) >= 1.21 for package: perl-IO-Socket-SSL-1.94-7.el7.noarch
--> Processing Dependency: perl(IO::Socket::IP) >= 0.20 for package: perl-IO-Socket-SSL-1.94-7.el7.noarch
--> Processing Dependency: perl(Net::SSLeay) for package: perl-IO-Socket-SSL-1.94-7.el7.noarch
--> Processing Dependency: perl(Net::LibIDN) for package: perl-IO-Socket-SSL-1.94-7.el7.noarch
--> Processing Dependency: perl(Mozilla::CA) for package: perl-IO-Socket-SSL-1.94-7.el7.noarch
--> Running transaction check
---> Package perl-IO-Socket-IP.noarch 0:0.21-5.el7 will be installed
---> Package perl-Mozilla-CA.noarch 0:20130114-5.el7 will be installed
---> Package perl-Net-LibIDN.x86_64 0:0.12-15.el7 will be installed
---> Package perl-Net-SSLeay.x86_64 0:1.55-6.el7 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
=========================================================================================================================================================================================
Package Arch Version Repository Size
=========================================================================================================================================================================================
Installing:
percona-toolkit x86_64 3.3.1-1.el7 /percona-toolkit-3.3.1-1.el7.x86_64 61 M
Installing for dependencies:
perl-IO-Socket-IP noarch 0.21-5.el7 c6-media 36 k
perl-IO-Socket-SSL noarch 1.94-7.el7 c6-media 114 k
perl-Mozilla-CA noarch 20130114-5.el7 c6-media 11 k
perl-Net-LibIDN x86_64 0.12-15.el7 c6-media 28 k
perl-Net-SSLeay x86_64 1.55-6.el7 c6-media 286 k
Transaction Summary
=========================================================================================================================================================================================
Install 1 Package (+5 Dependent packages)
Total size: 61 M
Total download size: 476 k
Installed size: 62 M
Is this ok [y/d/N]: y
Downloading packages:
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total 24 MB/s | 476 kB 00:00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : perl-Mozilla-CA-20130114-5.el7.noarch 1/6
Installing : perl-Net-LibIDN-0.12-15.el7.x86_64 2/6
Installing : perl-Net-SSLeay-1.55-6.el7.x86_64 3/6
Installing : perl-IO-Socket-IP-0.21-5.el7.noarch 4/6
Installing : perl-IO-Socket-SSL-1.94-7.el7.noarch 5/6
Installing : percona-toolkit-3.3.1-1.el7.x86_64 6/6
Verifying : perl-IO-Socket-IP-0.21-5.el7.noarch 1/6
Verifying : perl-Net-SSLeay-1.55-6.el7.x86_64 2/6
Verifying : percona-toolkit-3.3.1-1.el7.x86_64 3/6
Verifying : perl-Net-LibIDN-0.12-15.el7.x86_64 4/6
Verifying : perl-IO-Socket-SSL-1.94-7.el7.noarch 5/6
Verifying : perl-Mozilla-CA-20130114-5.el7.noarch 6/6
Installed:
percona-toolkit.x86_64 0:3.3.1-1.el7
Dependency Installed:
perl-IO-Socket-IP.noarch 0:0.21-5.el7 perl-IO-Socket-SSL.noarch 0:1.94-7.el7 perl-Mozilla-CA.noarch 0:20130114-5.el7 perl-Net-LibIDN.x86_64 0:0.12-15.el7
perl-Net-SSLeay.x86_64 0:1.55-6.el7
Complete!
查看安装位置：
rpm -qa|grep percona
rpm -ql percona-toolkit-3.3.1-1.el7.x86_64
[root@test28 local]# rpm -qa|grep percona
percona-xtrabackup-80-8.0.26-18.1.el7.x86_64
percona-toolkit-3.3.1-1.el7.x86_64
[root@test28 local]# rpm -ql percona-toolkit-3.3.1-1.el7.x86_64
/usr/bin/pt-align
/usr/bin/pt-archiver
/usr/bin/pt-config-diff
/usr/bin/pt-deadlock-logger
/usr/bin/pt-diskstats
/usr/bin/pt-duplicate-key-checker
/usr/bin/pt-fifo-split
/usr/bin/pt-find
/usr/bin/pt-fingerprint
/usr/bin/pt-fk-error-logger
/usr/bin/pt-heartbeat
/usr/bin/pt-index-usage
/usr/bin/pt-ioprofile
/usr/bin/pt-k8s-debug-collector
/usr/bin/pt-kill
/usr/bin/pt-mext
/usr/bin/pt-mongodb-query-digest
/usr/bin/pt-mongodb-summary
/usr/bin/pt-mysql-summary
/usr/bin/pt-online-schema-change
/usr/bin/pt-pg-summary
/usr/bin/pt-pmp
/usr/bin/pt-query-digest
/usr/bin/pt-secure-collect
/usr/bin/pt-show-grants
/usr/bin/pt-sift
/usr/bin/pt-slave-delay
/usr/bin/pt-slave-find
/usr/bin/pt-slave-restart
/usr/bin/pt-stalk
/usr/bin/pt-summary
/usr/bin/pt-table-checksum
/usr/bin/pt-table-sync
/usr/bin/pt-table-usage
/usr/bin/pt-upgrade
/usr/bin/pt-variable-advisor
/usr/bin/pt-visual-explain
/usr/share/doc/percona-toolkit-3.3.1
/usr/share/doc/percona-toolkit-3.3.1/COPYING
/usr/share/doc/percona-toolkit-3.3.1/Changelog
/usr/share/doc/percona-toolkit-3.3.1/INSTALL
/usr/share/doc/percona-toolkit-3.3.1/README.md
/usr/share/man/man1/percona-toolkit.1p.gz
/usr/share/man/man1/pt-align.1p.gz
/usr/share/man/man1/pt-archiver.1p.gz
/usr/share/man/man1/pt-config-diff.1p.gz
/usr/share/man/man1/pt-deadlock-logger.1p.gz
/usr/share/man/man1/pt-diskstats.1p.gz
/usr/share/man/man1/pt-duplicate-key-checker.1p.gz
/usr/share/man/man1/pt-fifo-split.1p.gz
/usr/share/man/man1/pt-find.1p.gz
/usr/share/man/man1/pt-fingerprint.1p.gz
/usr/share/man/man1/pt-fk-error-logger.1p.gz
/usr/share/man/man1/pt-heartbeat.1p.gz
/usr/share/man/man1/pt-index-usage.1p.gz
/usr/share/man/man1/pt-ioprofile.1p.gz
/usr/share/man/man1/pt-k8s-debug-collector.1p.gz
/usr/share/man/man1/pt-kill.1p.gz
/usr/share/man/man1/pt-mext.1p.gz
/usr/share/man/man1/pt-mysql-summary.1p.gz
/usr/share/man/man1/pt-online-schema-change.1p.gz
/usr/share/man/man1/pt-pg-summary.1p.gz
/usr/share/man/man1/pt-pmp.1p.gz
/usr/share/man/man1/pt-query-digest.1p.gz
/usr/share/man/man1/pt-secure-collect.1p.gz
/usr/share/man/man1/pt-show-grants.1p.gz
/usr/share/man/man1/pt-sift.1p.gz
/usr/share/man/man1/pt-slave-delay.1p.gz
/usr/share/man/man1/pt-slave-find.1p.gz
/usr/share/man/man1/pt-slave-restart.1p.gz
/usr/share/man/man1/pt-stalk.1p.gz
/usr/share/man/man1/pt-summary.1p.gz
/usr/share/man/man1/pt-table-checksum.1p.gz
/usr/share/man/man1/pt-table-sync.1p.gz
/usr/share/man/man1/pt-table-usage.1p.gz
/usr/share/man/man1/pt-upgrade.1p.gz
/usr/share/man/man1/pt-variable-advisor.1p.gz
/usr/share/man/man1/pt-visual-explain.1p.gz
[root@test28 bin]# yum provides pt-table-sync
Loaded plugins: product-id, search-disabled-repos, subscription-manager
This system is not registered with an entitlement server. You can use subscription-manager to register.
c6-media 5230/5230
c6-media/filelists | 3.1 MB 00:00:00
percona-toolkit-3.3.1-1.el7.x86_64 : Advanced MySQL and system command-line tools
Repo : @/percona-toolkit-3.3.1-1.el7.x86_64
Matched from:
Filename : /usr/bin/pt-table-sync
就在yum的默认安装位置。
pt打头的都是安装好的工具：
[root@test28 bin]# ls -ltr pt*
-rwxr-xr-x. 1 root root 3891 Jul 6 2016 ptaskset
-rwxr-xr-x. 1 root root 66720 Mar 16 2019 ptx
-rwxr-xr-x. 1 root root 437836 Apr 23 2021 pt-online-schema-change
-rwxr-xr-x. 1 root root 108113 Apr 23 2021 pt-mysql-summary
-rwxr-xr-x. 1 root root 257732 Apr 23 2021 pt-kill
-rwxr-xr-x. 1 root root 103460 Apr 23 2021 pt-visual-explain
-rwxr-xr-x. 1 root root 179330 Apr 23 2021 pt-variable-advisor
-rwxr-xr-x. 1 root root 334032 Apr 23 2021 pt-upgrade
-rwxr-xr-x. 1 root root 248658 Apr 23 2021 pt-table-usage
-rwxr-xr-x. 1 root root 406401 Apr 23 2021 pt-table-sync
-rwxr-xr-x. 1 root root 461940 Apr 23 2021 pt-table-checksum
-rwxr-xr-x. 1 root root 90817 Apr 23 2021 pt-summary
-rwxr-xr-x. 1 root root 76186 Apr 23 2021 pt-stalk
-rwxr-xr-x. 1 root root 186580 Apr 23 2021 pt-slave-restart
-rwxr-xr-x. 1 root root 133040 Apr 23 2021 pt-slave-find
-rwxr-xr-x. 1 root root 147869 Apr 23 2021 pt-slave-delay
-rwxr-xr-x. 1 root root 37784 Apr 23 2021 pt-sift
-rwxr-xr-x. 1 root root 79159 Apr 23 2021 pt-show-grants
-rwxr-xr-x. 1 root root 529350 Apr 23 2021 pt-query-digest
-rwxr-xr-x. 1 root root 24598 Apr 23 2021 pt-pmp
-rwxr-xr-x. 1 root root 21913 Apr 23 2021 pt-mext
-rwxr-xr-x. 1 root root 32405 Apr 23 2021 pt-ioprofile
-rwxr-xr-x. 1 root root 229128 Apr 23 2021 pt-index-usage
-rwxr-xr-x. 1 root root 225530 Apr 23 2021 pt-heartbeat
-rwxr-xr-x. 1 root root 135870 Apr 23 2021 pt-fk-error-logger
-rwxr-xr-x. 1 root root 67304 Apr 23 2021 pt-fingerprint
-rwxr-xr-x. 1 root root 152724 Apr 23 2021 pt-find
-rwxr-xr-x. 1 root root 50157 Apr 23 2021 pt-fifo-split
-rwxr-xr-x. 1 root root 172014 Apr 23 2021 pt-duplicate-key-checker
-rwxr-xr-x. 1 root root 166450 Apr 23 2021 pt-diskstats
-rwxr-xr-x. 1 root root 168891 Apr 23 2021 pt-deadlock-logger
-rwxr-xr-x. 1 root root 171696 Apr 23 2021 pt-config-diff
-rwxr-xr-x. 1 root root 272712 Apr 23 2021 pt-archiver
-rwxr-xr-x. 1 root root 41747 Apr 23 2021 pt-align
-rwxr-xr-x. 1 root root 4795128 Apr 23 2021 pt-secure-collect
-rwxr-xr-x. 1 root root 6728024 Apr 23 2021 pt-pg-summary
-rwxr-xr-x. 1 root root 12591128 Apr 23 2021 pt-mongodb-summary
-rwxr-xr-x. 1 root root 12212984 Apr 23 2021 pt-mongodb-query-digest
-rwxr-xr-x. 1 root root 13534872 Apr 23 2021 pt-k8s-debug-collector
数据校验
初试
创建连接用户：
root@localhost 11:57: [(none)]> create user pt identified by 'pt';
Query OK, 0 rows affected (0.02 sec)
root@localhost 11:58: [(none)]> grant all privileges on *.* to pt;
Query OK, 0 rows affected (0.01 sec)
连接测试
[root@test28 logs]# pt-table-checksum --recursion-method="processlist" \\
> --replicate="zhuo.checksums" \\
> --host='192.168.33.28' \\
> --port=3306 \\
> --user='pt' \\
> --password='pt' \\
> --databases=sbtest \\
> --no-check-binlog-format
01-20T12:00:59 DBI connect(';host=192.168.33.28;port=3306;mysql_read_default_group=client','pt',...) failed: Authentication plugin 'caching_sha2_password' cannot be loaded: /usr/lib64/mysql/plugin/caching_sha2_password.so: cannot open shared object file: No such file or directory at /usr/bin/pt-table-checksum line 1623.
报错：Authentication plugin ‘caching_sha2_password’ cannot be loaded 密码验证方式不兼容。
前面也经常遇到，修改此用户的密码验证方式为：mysql_native_password 即可。
修改如下：
root@localhost 12:04: [(none)]> ALTER USER 'pt'@'%' IDENTIFIED WITH mysql_native_password BY 'pt';
Query OK, 0 rows affected (0.00 sec)
root@localhost 12:06: [(none)]> select user,host,plugin from mysql.user;
+------------------+-----------+-----------------------+
| user | host | plugin |
+------------------+-----------+-----------------------+
| pt | % | mysql_native_password |
| repl | % | caching_sha2_password |
| wahaha | % | caching_sha2_password |
| zhuo | % | caching_sha2_password |
| mysql.infoschema | localhost | caching_sha2_password |
| mysql.session | localhost | caching_sha2_password |
| mysql.sys | localhost | caching_sha2_password |
| root | localhost | caching_sha2_password |
+------------------+-----------+-----------------------+
9 rows in set (0.00 sec)
也可以以后直接创建用户的时候，直接修改密码验证方式，如下：
root@localhost 12:06: [(none)]> create user test identified WITH mysql_native_password by 'test' ;
Query OK, 0 rows affected (0.01 sec)
root@localhost 12:06: [(none)]> select user,host,plugin from mysql.user;
+------------------+-----------+-----------------------+
| user | host | plugin |
+------------------+-----------+-----------------------+
| pt | % | mysql_native_password |
| repl | % | caching_sha2_password |
| test | % | mysql_native_password |
| wahaha | % | caching_sha2_password |
| zhuo | % | caching_sha2_password |
| mysql.infoschema | localhost | caching_sha2_password |
| mysql.session | localhost | caching_sha2_password |
| mysql.sys | localhost | caching_sha2_password |
| root | localhost | caching_sha2_password |
+------------------+-----------+-----------------------+
9 rows in set (0.00 sec)
[root@test28 logs]# pt-table-checksum --recursion-method="processlist" --replicate="zhuo.checksums" --host='192.168.33.28' --port=3306 --user='pt' --password='pt' --databases=sbtest --n
o-check-binlog-format
Checking if all tables can be checksummed ...
Starting checksum ...
Diffs cannot be detected because no slaves were found. Please read the --recursion-method documentation for information.
TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE
01-20T12:07:15 0 0 1098156 0 15 0 5.856 sbtest.sbtest1
01-20T12:07:20 0 0 1099426 0 1 0 5.216 sbtest.sbtest10
01-20T12:07:26 0 0 1100876 0 1 0 5.454 sbtest.sbtest2
01-20T12:07:31 0 0 1099857 0 1 0 4.912 sbtest.sbtest3
01-20T12:07:36 0 0 1099689 0 1 0 5.210 sbtest.sbtest4
01-20T12:07:41 0 0 1100036 0 1 0 5.080 sbtest.sbtest5
01-20T12:07:47 0 0 1098960 0 1 0 5.517 sbtest.sbtest6
01-20T12:07:52 0 0 1098998 0 1 0 5.452 sbtest.sbtest7
01-20T12:07:57 0 0 1099965 0 1 0 5.001 sbtest.sbtest8
01-20T12:08:02 0 0 1098637 0 1 0 5.037 sbtest.sbtest9
报错：
Diffs cannot be detected because no slaves were found.
主库找不到从库。让添加参数：–recursion-method
通过
[root@test28 ~]# pt-table-checksum --help
参数如下解释：
–recursion-method=a Preferred recursion method for discovering
replicas. pt-table-checksum performs several
REPLICA CHECKS before and while running (
default processlist,hosts)
默认是processlist和hosts在主库查询slave的信息。
[root@test28 ~]# pt-table-checksum --replicate="zhuo.checksums" --host='192.168.33.28' --port=3306 --user='pt' --password='pt' --databases=sbtest --no-check-binlog-format
Checking if all tables can be checksummed ...
Starting checksum ...
Diffs cannot be detected because no slaves were found. Please read the --recursion-method documentation for information.
TS ERRORS DIFFS ROWS DIFF_ROWS CHUNKS SKIPPED TIME TABLE
01-21T00:33:31 0 0 10000 0 4 0 0.110 sbtest.sbtest1
01-21T00:33:31 0 0 10000 0 1 0 0.067 sbtest.sbtest10
01-21T00:33:32 0 0 10000 0 1 0 0.067 sbtest.sbtest2
01-21T00:33:32 0 0 10000 0 1 0 0.066 sbtest.sbtest3
01-21T00:33:32 0 0 10000 0 1 0 0.070 sbtest.sbtest4
01-21T00:33:32 0 0 10000 0 1 0 0.067 sbtest.sbtest5
01-21T00:33:32 0 0 10000 0 1 0 0.067 sbtest.sbtest6
01-21T00:33:32 0 0 10000 0 1 0 0.067 sbtest.sbtest7
01-21T00:33:32 0 0 10000 0 1 0 0.067 sbtest.sbtest8
01-21T00:33:32 0 0 10000 0 1 0 0.068 sbtest.sbtest9
还是报相同错误，我们手动在主库查询，看能否查询到从库的信息。
主库查询：
root@localhost 00:28: [(none)]> show processlist;
+-----+-----------------+--------------+------+-------------+--------+-----------------------------------------------------------------+------------------+
| Id | User | Host | db | Command | Time | State | Info |
+-----+-----------------+--------------+------+-------------+--------+-----------------------------------------------------------------+------------------+
| 5 | event_scheduler | localhost | NULL | Daemon | 646933 | Waiting on empty queue | NULL |
| 922 | repl | test28:38170 | NULL | Binlog Dump | 2010 | Source has sent all binlog to replica; waiting for more updates | NULL |
| 925 | root | localhost | NULL | Query | 0 | init | show processlist |
+-----+-----------------+--------------+------+-------------+--------+-----------------------------------------------------------------+------------------+
3 rows in set (0.00 sec)
root@localhost 00:28: [(none)]> show slave hosts;
+-----------+---------------+------+-----------+--------------------------------------+
| Server_id | Host | Port | Master_id | Slave_UUID |
+-----------+---------------+------+-----------+--------------------------------------+
| 283310 | 192.168.33.28 | 3310 | 3306 | 6bbd5c37-761e-11ec-b67a-0050569e5ca3 |
+-----------+---------------+------+-----------+--------------------------------------+
1 row in set, 1 warning (0.00 sec)
是可以查询到的。
网上有些说的，当在主库show查询不到从库的信息的时候，需要在从库的my.cnf里面添加从库自己的信息，即：
report_host = MASTER_HOST
report_port = 13306
具体为：
[mysqld]
report-host=192.168.33.28
report-port =3306
重启从库。亲测，无效。
他们这种都是针对主库show查询不到从库信息的方案，我们此处是能查询到的，所以不适用。
看到如下原因：
pt-table-checksum 默认可以在主库的 processlist 中找到从库复制进程，从而识别出有哪些从库，但如果使用是非标准3306端口，会导致找不到从库信息。意思show processlist只能检测使用3306默认端口的从库。所以此处我们检测不到从库。以上是关于pt-table-checksum的主要内容，如果未能解决你的问题，请参考以下文章