mysql主从之半同步复制和lossless无损复制
Posted zyxnhr
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了mysql主从之半同步复制和lossless无损复制相关的知识，希望对你有一定的参考价值。
一 mysql 的三种复制方式
1.1 简介
asynchronous 异步复制
fully synchronous 全同步复制
Semisynchronous 半同步复制
从MySQL5.5 开始，MySQL 以插件的形式支持半同步复制。
1.2 异步复制（Asynchronous replication）
MySQL 默认的复制是异步的，主库在执行完客户端提交的事务后会立即将结果返给给客户端，并不关心从库是否已经接收并处理，这样就会有一个问题，主如果crash 掉了，此时主上已经提交的事务可能并没有传到从上，如果此时，将从提升为主，可能导致新主上的数据不完整。
原理：在异步复制中，master 写数据到binlog 且sync，slave request binlog 后写入relay‐log 并flush disk
优点：复制的性能最好
缺点：master 挂掉后，slave 可能会丢失数据
1.3 全同步复制（Fully synchronous replication）
指当主库执行完一个事务，所有的从库都执行了该事务才返回给客户端。因为需要等待所有从库执行完该事务才能返回，所以全同步复制的性能必然会收到严重的影响。
优点：数据不会丢失
缺点：会阻塞master session，性能太差，非常依赖网络
1.4 半同步复制（Semisynchronous replication）
介于异步复制和全同步复制之间，主库在执行完客户端提交的事务后不是立刻返回给客户端，而是等待至少一个从库接收到并写到relay log 中才返回给客户端。相对于异步复制，半同步复制提高了数据的安全性，同时它也造成了一定程度的延迟，这个延迟最少是一个TCP/IP 往返的时间。所以，半同步复制最好在低延时的网络中使用。
优点：会有数据丢失风险（低）
缺点：会阻塞master session，性能差，非常依赖网络，由于master 是在三段提交的最后commit 阶段完成后才等待，所以master 的其他session 是可以看到这个提交事务的，所以这时候master 上的数据和slave 不一致，master crash 后，slave 数据丢失。
1.5 增强版的半同步复制（lossless replication）
原理: 在半同步复制中，master 写数据到binlog 且sync，然后一直等待ACK. 当至少一个slave request bilog 后写入到relay‐log 并flush disk，就返回ack（不需要回放完日志）
优点：数据零丢失（前提是让其一直是lossless replication），
性能好
缺点：会阻塞master session，非常依赖网络由于master 是在三段提交的第二阶段sync binlog 完成后才等待, 所以master 的其他session 是看不见这个提交事务的，所以这时候master 上的数据和slave 一致，master crash 后，slave 没有丢失数据
二 实验
2.1 查看plugins
mysql> show plugins;
+----------------------------+----------+--------------------+---------+---------+
| Name | Status | Type | Library | License |
+----------------------------+----------+--------------------+---------+---------+
| binlog | ACTIVE | STORAGE ENGINE | NULL | GPL |
| mysql_native_password | ACTIVE | AUTHENTICATION | NULL | GPL |
| sha256_password | ACTIVE | AUTHENTICATION | NULL | GPL |
| CSV | ACTIVE | STORAGE ENGINE | NULL | GPL |
| MEMORY | ACTIVE | STORAGE ENGINE | NULL | GPL |
| InnoDB | ACTIVE | STORAGE ENGINE | NULL | GPL |
| INNODB_TRX | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_LOCKS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_LOCK_WAITS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMP | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMP_RESET | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMPMEM | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMPMEM_RESET | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMP_PER_INDEX | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_CMP_PER_INDEX_RESET | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_BUFFER_PAGE | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_BUFFER_PAGE_LRU | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_BUFFER_POOL_STATS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_TEMP_TABLE_INFO | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_METRICS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_DEFAULT_STOPWORD | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_DELETED | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_BEING_DELETED | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_CONFIG | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_INDEX_CACHE | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_FT_INDEX_TABLE | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_TABLES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_TABLESTATS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_INDEXES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_COLUMNS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_FIELDS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_FOREIGN | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_FOREIGN_COLS | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_TABLESPACES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_DATAFILES | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| INNODB_SYS_VIRTUAL | ACTIVE | INFORMATION SCHEMA | NULL | GPL |
| MyISAM | ACTIVE | STORAGE ENGINE | NULL | GPL |
| MRG_MYISAM | ACTIVE | STORAGE ENGINE | NULL | GPL |
| PERFORMANCE_SCHEMA | ACTIVE | STORAGE ENGINE | NULL | GPL |
| ARCHIVE | ACTIVE | STORAGE ENGINE | NULL | GPL |
| BLACKHOLE | ACTIVE | STORAGE ENGINE | NULL | GPL |
| FEDERATED | DISABLED | STORAGE ENGINE | NULL | GPL |
| partition | ACTIVE | STORAGE ENGINE | NULL | GPL |
| ngram | ACTIVE | FTPARSER | NULL | GPL |
+----------------------------+----------+--------------------+---------+---------+
或者查询INFORMATION_SCHEMA.PLUGINS 表
[[email protected] ~]# cd /usr/lib64/mysql/plugin/
[[email protected] plugin]# ll
-rwxr-xr-x. 1 root root 103728 Apr 13 10:36 adt_null.so -rwxr-xr-x. 1 root root 356976 Apr 13 10:36 authentication_ldap_sasl_client.so -rwxr-xr-x. 1 root root 43552 Apr 13 10:36 auth_socket.so -rwxr-xr-x. 1 root root 940312 Apr 13 10:36 connection_control.so drwxr-xr-x. 2 root root 4096 Jul 3 11:31 debug -rwxr-xr-x. 1 root root 21640312 Apr 13 10:36 group_replication.so -rwxr-xr-x. 1 root root 483520 Apr 13 10:36 ha_example.so -rwxr-xr-x. 1 root root 968432 Apr 13 10:36 innodb_engine.so -rwxr-xr-x. 1 root root 957088 Apr 13 10:36 keyring_file.so -rwxr-xr-x. 1 root root 460064 Apr 13 10:36 keyring_udf.so -rwxr-xr-x. 1 root root 1184680 Apr 13 10:36 libmemcached.so -rwxr-xr-x. 1 root root 8973984 Apr 13 10:36 libpluginmecab.so -rwxr-xr-x. 1 root root 21424 Apr 13 10:36 locking_service.so -rwxr-xr-x. 1 root root 53928 Apr 13 10:36 mypluglib.so -rwxr-xr-x. 1 root root 41088 Apr 13 10:36 mysql_no_login.so -rwxr-xr-x. 1 root root 22243648 Apr 13 10:37 mysqlx.so -rwxr-xr-x. 1 root root 49504 Apr 13 10:36 rewrite_example.so -rwxr-xr-x. 1 root root 590936 Apr 13 10:36 rewriter.so -rwxr-xr-x. 1 root root 933904 Apr 13 10:36 semisync_master.so #主库安装 -rwxr-xr-x. 1 root root 159928 Apr 13 10:36 semisync_slave.so #备库安装 -rwxr-xr-x. 1 root root 209520 Apr 13 10:36 validate_password.so -rwxr-xr-x. 1 root root 506320 Apr 13 10:36 version_token.so
2.2 主库配置
查看是否支持动态加载的MySQL 服务器
mysql> show variables like ‘%dynamic%‘; +----------------------+-------+ | Variable_name | Value | +----------------------+-------+ | have_dynamic_loading | YES | +----------------------+-------+ 1 row in set (0.00 sec) mysql> install plugin rpl_semi_sync_master soname ‘semisync_master.so‘; #安装库 Query OK, 0 rows affected (0.00 sec) mysql> show variables like ‘%rpl_semi%‘; +-------------------------------------------+------------+ | Variable_name | Value | +-------------------------------------------+------------+ | rpl_semi_sync_master_enabled | OFF | #修改为on状态 | rpl_semi_sync_master_timeout | 10000 | #修改为1s | rpl_semi_sync_master_trace_level | 32 | | rpl_semi_sync_master_wait_for_slave_count | 1 | | rpl_semi_sync_master_wait_no_slave | ON | | rpl_semi_sync_master_wait_point | AFTER_SYNC | +-------------------------------------------+------------+
修改my.cnf
[[email protected] ~]# vim /etc/my.cnf
rpl_semi_sync_master_enabled = 1 rpl_semi_sync_master_timeout = 1000
[[email protected] ~]# systemctl restart mysqld
mysql> show variables like ‘%rpl_semi%‘; +-------------------------------------------+------------+ | Variable_name | Value | +-------------------------------------------+------------+ | rpl_semi_sync_master_enabled | ON | | rpl_semi_sync_master_timeout | 1000 | | rpl_semi_sync_master_trace_level | 32 | | rpl_semi_sync_master_wait_for_slave_count | 1 | | rpl_semi_sync_master_wait_no_slave | ON | | rpl_semi_sync_master_wait_point | AFTER_SYNC | +-------------------------------------------+------------+ mysql> show status like ‘%semi%‘; +--------------------------------------------+-------+ | Variable_name | Value | +--------------------------------------------+-------+ | Rpl_semi_sync_master_clients | 0 | | Rpl_semi_sync_master_net_avg_wait_time | 0 | | Rpl_semi_sync_master_net_wait_time | 0 | | Rpl_semi_sync_master_net_waits | 0 | | Rpl_semi_sync_master_no_times | 0 | | Rpl_semi_sync_master_no_tx | 0 | | Rpl_semi_sync_master_status | ON | | Rpl_semi_sync_master_timefunc_failures | 0 | | Rpl_semi_sync_master_tx_avg_wait_time | 0 | | Rpl_semi_sync_master_tx_wait_time | 0 | | Rpl_semi_sync_master_tx_waits | 0 | | Rpl_semi_sync_master_wait_pos_backtraverse | 0 | | Rpl_semi_sync_master_wait_sessions | 0 | | Rpl_semi_sync_master_yes_tx | 0 | +--------------------------------------------+-------+
rpl_semi_sync_master_timeout
一个以毫秒为单位的值，用于控制主服务器等待来自从服务器的确认提交并恢复到异步复制的时间，超过这个值就是超时。 默认值是10000（10 秒）。超时之后，就从半同步复制，返回到异步复制。
Rpl_semi_sync_master_yes_tx:从库成功确认的提交数量。
Rpl_semi_sync_master_no_tx:从库未成功确认的提交数量。
2.3 备份服务器配置
mysql> show variables like ‘%dynamic%‘; +----------------------+-------+ | Variable_name | Value | +----------------------+-------+ | have_dynamic_loading | YES | +----------------------+-------+ mysql> install plugin rpl_semi_sync_slave soname ‘semisync_slave.so‘; mysql> show variables like ‘%rpl_semi%‘; +---------------------------------+-------+ | Variable_name | Value | +---------------------------------+-------+ | rpl_semi_sync_slave_enabled | OFF | #打开为on | rpl_semi_sync_slave_trace_level | 32 | +---------------------------------+-------+
修改my.cnf
[[email protected] ~]# vim /etc/my.cnf
rpl_semi_sync_slave_enabled = 1
[[email protected] ~]# systemctl restart mysqld
mysql> show variables like ‘%rpl_semi%‘; +---------------------------------+-------+ | Variable_name | Value | +---------------------------------+-------+ | rpl_semi_sync_slave_enabled | ON | | rpl_semi_sync_slave_trace_level | 32 | +---------------------------------+-------+ mysql> show status like ‘%semi%‘; +----------------------------+-------+ | Variable_name | Value | +----------------------------+-------+ | Rpl_semi_sync_slave_status | ON | +----------------------------+-------+
2.4 主库验证
mysql> show status like ‘%semi%‘; +--------------------------------------------+-------+ | Variable_name | Value | +--------------------------------------------+-------+ | Rpl_semi_sync_master_clients | 1 | | Rpl_semi_sync_master_net_avg_wait_time | 0 | | Rpl_semi_sync_master_net_wait_time | 0 | | Rpl_semi_sync_master_net_waits | 2 | | Rpl_semi_sync_master_no_times | 1 | | Rpl_semi_sync_master_no_tx | 1 | | Rpl_semi_sync_master_status | ON | | Rpl_semi_sync_master_timefunc_failures | 0 | | Rpl_semi_sync_master_tx_avg_wait_time | 0 | | Rpl_semi_sync_master_tx_wait_time | 0 | | Rpl_semi_sync_master_tx_waits | 0 | | Rpl_semi_sync_master_wait_pos_backtraverse | 0 | | Rpl_semi_sync_master_wait_sessions | 0 | | Rpl_semi_sync_master_yes_tx | 0 | +--------------------------------------------+-------+ mysql> insert into test values (2); mysql> insert into test values (2); mysql> insert into test values (2); mysql> show status like ‘%semi%‘; +--------------------------------------------+-------+ | Variable_name | Value | +--------------------------------------------+-------+ | Rpl_semi_sync_master_clients | 1 | | Rpl_semi_sync_master_net_avg_wait_time | 0 | | Rpl_semi_sync_master_net_wait_time | 0 | | Rpl_semi_sync_master_net_waits | 5 | | Rpl_semi_sync_master_no_times | 1 | | Rpl_semi_sync_master_no_tx | 1 | | Rpl_semi_sync_master_status | ON | | Rpl_semi_sync_master_timefunc_failures | 0 | | Rpl_semi_sync_master_tx_avg_wait_time | 1013 | | Rpl_semi_sync_master_tx_wait_time | 3041 | | Rpl_semi_sync_master_tx_waits | 3 | | Rpl_semi_sync_master_wait_pos_backtraverse | 0 | | Rpl_semi_sync_master_wait_sessions | 0 | | Rpl_semi_sync_master_yes_tx | 3 | #增加数据.这个之会增加 +--------------------------------------------+-------+
从端数据已经同步
mysql> select * from master1.test; +------+ | id | +------+ | 1 | | 2 | | 2 | | 2 | +------+
2.5 测试AFTER_SYNC 和AFTER_COMMIT
主库设置超时时间为1000 秒，备库停掉复制，模拟timeout
mysql> set global rpl_semi_sync_master_timeout=1000000; mysql> stop slave; mysql> insert into test values (11); #会一直卡住 mysql> select * from master1.test; +------+ | id | +------+ | 1 | | 2 | | 2 | | 2 | +------+
重启主库数据库,模拟主库宕机,从看数据记录
[[email protected] ~]# systemctl start mysqld [[email protected] ~]# mysql -uroot -p123456 mysql> select * from master1.test; +------+ | id | +------+ | 1 | | 2 | | 2 | | 2 | | 11 | #主库有记录 +------+
备库开启slave
mysql> start slave; Query OK, 0 rows affected (0.00 sec) mysql> select * from master1.test; +------+ | id | +------+ | 1 | | 2 | | 2 | | 2 | | 11 | #数据已经同步,没有丢失 +------+
无损的半同步复制是在write binlog 之后。需要得到备库的确认。所以这时候主库宕机，不会发生丢数据。当主库启动后，插入的数据重新可见。
将rpl_semi_sync_master_wait_point 设置为AFTER_COMMIT，
再次测试:
主库设置超时时间为1000 秒，备库停掉复制，模拟timeout
mysql> set global rpl_semi_sync_master_wait_point=AFTER_COMMIT; mysql> set global rpl_semi_sync_master_timeout=1000000; mysql> show variables like ‘%semi%‘; +-------------------------------------------+--------------+ | Variable_name | Value | +-------------------------------------------+--------------+ | rpl_semi_sync_master_enabled | ON | | rpl_semi_sync_master_timeout | 1000000 | | rpl_semi_sync_master_trace_level | 32 | | rpl_semi_sync_master_wait_for_slave_count | 1 | | rpl_semi_sync_master_wait_no_slave | ON | | rpl_semi_sync_master_wait_point | AFTER_COMMIT | +-------------------------------------------+--------------+
mysql> insert into master1.test values (12); #一直卡住
[[email protected] ~]# mysql -uroot -p123456 #另开一个窗口,发现已经有12这个数据 mysql> select * from master1.test; +------+ | id | +------+ | 1 | | 2 | | 2 | | 2 | | 11 | | 12 | +------+
这样当从库起来之后,数据已经提交,从库就会缺少这个数据
再开一个窗口查询这条数据，发现可以查询到。这时候主库宕机，会发生数据丢失。
主库重新启动，备库启动slave 会同步到备库。
以上是关于mysql主从之半同步复制和lossless无损复制的主要内容，如果未能解决你的问题，请参考以下文章