mysql的主从主主及半同步复制
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了mysql的主从主主及半同步复制相关的知识，希望对你有一定的参考价值。
mysql复制
Mysql内建的复制功能是构建大型，高性能应用程序的基础。将Mysql的数据分布到多个系统上去，这种分布的机制，是通过将Mysql的某一台主机的数据复制到其它主机（slaves）上，并重新执行一遍来实现的。复制过程中一个服务器充当主服务器，而一个或多个其它服务器充当从服务器。主服务器将更新写入二进制日志文件，并维护文件的一个索引以跟踪日志循环。这些日志可以记录发送到从服务器的更新。当一个从服务器连接主服务器时，它通知主服务器从服务器在日志中读取的最后一次成功更新的位置。从服务器接收从那时起发生的任何更新，然后封锁并等待主服务器通知新的更新。
复制如何工作
整体上来说，复制有3个步骤：
(1) master将改变记录到二进制日志(binary log)中（这些记录叫做二进制日志事件，binary log events）；
(2) slave将master的binary log events拷贝到它的中继日志(relay log)；
(3) slave重做中继日志中的事件，将改变反映它自己的数据。
一、mysql的主从复制模型
主节点：启用sqldump线程，启用二进制日志，接收从节点端的线程请求，并检索本地二进制日志是否有新的日志产生，如果有，就从二进制日志中读取并响应给从节点端的线程。
从节点：当成Mysql客户端，并启用I/O Thread和SQL Thread两线程。I/O线程去请求主服务上的二进制日志，并将请求到的二进制日志写入到中继日志中，SQL线程通过读取中继日志的日志事件，在本地完成重放。
配置主节点：
[mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock symbolic-links=0 log-bin=master-bin //启用二进制日志 innodb_file_per_table=ON //如果是MyISAM引擎，可以不用写 server_id=1 //开启唯一ID号 MariaDB [(none)]> grant replication slave,replication client on *.* to ‘admin‘@‘172.18.250.78‘ identified by ‘admin‘; //授权从服务器复制权限的账号 MariaDB [(none)]> show master status; +-------------------+----------+--------------+------------------+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +-------------------+----------+--------------+------------------+ | master-bin.000003 | 418 | | | +-------------------+----------+--------------+------------------+ 1 row in set (0.01 sec) //显示主服务器正在使用的二进制而知文件和事件的偏移量
配置从节点：
[mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock symbolic-links=0 skip_name_resolve=ON innodb_file_per_table=ON relay-log=relay-log //开启中继日志 server_id=2 //开启唯一ID号 read-only=1 //从服务只读，不执行写操作 MariaDB [(none)]> help change master to; CHANGE MASTER TO option [, option] ... option: | MASTER_HOST = ‘host_name‘ //主服务IP或主机名 | MASTER_USER = ‘user_name‘ //具有复制权限的账号 | MASTER_PASSWORD = ‘password‘ //具有复制权限的密码 | MASTER_PORT = port_num //数据库端口，默认是3306可以省略 | MASTER_CONNECT_RETRY = interval //无法连接主服务器的时候可以重试连接多少次 | MASTER_HEARTBEAT_PERIOD = interval //多长时间探测一次主服务器是否在线 | MASTER_LOG_FILE = ‘master_log_name‘ //从哪个二进制日志开始复制 | MASTER_LOG_POS = master_log_pos //指定二进制日志事件的偏移量 MariaDB [(none)]> change master to master_host=‘172.18.250.77‘, master_user=‘admin‘, master_password=‘admin‘, master_log_file=‘master-bin.000003‘, master_log_pos=418, master_connect_retry=10, MASTER_HEARTBEAT_PERIOD=5; MariaDB [(none)]> start slave; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> show slave status\G; *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.18.250.77 Master_User: admin Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000003 Read_Master_Log_Pos: 418 Relay_Log_File: relay-log.000002 Relay_Log_Pos: 530 Relay_Master_Log_File: master-bin.000003 Slave_IO_Running: Yes //这两个显示yes就表示主从复制正常 Slave_SQL_Running: Yes
测试主从同步是否正常：
MariaDB [(none)]> create database mytest; //主服务器是创建一个数据库 Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> show databases; //从服务器上查看 +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | mytest | //同步正常 | performance_schema | | test | +--------------------+ 5 rows in set (0.00 sec)
mysq进行复制时是通过明文进行传输的，别人可以通过中途截取明文数据就能知道公司的数据机密，尤其是跨互联网，跨机房进行复制时，所以传输过程中是相当不安全的，为了数据的安全，可以基于SSL进行复制传输
MariaDB [(none)]> show global variables like "%ssl%"; +---------------+----------+ | Variable_name | Value | +---------------+----------+ | have_openssl | DISABLED | //默认是关闭的，需要开启 | have_ssl | DISABLED | | ssl_ca | | | ssl_capath | | | ssl_cert | | | ssl_cipher | | | ssl_key | | +---------------+----------+ 7 rows in set (0.01 sec) 主节点上配置SSL： ]# vim /etc/my.cnf ssl //开启ssl ssl_ca=/var/lib/mysql/ssl/cacert.pem //ca证书 ssl_cert=/var/lib/mysql/ssl/mysql.crt //主节点证书 ssl_key=/var/lib/mysql/ssl/mysql.key //主节点私钥 ]# mkdir /var/lib/mysql/ssl ]# ll //根据openssl签署三个文件并放到这目录下 total 16 -rw-r--r-- 1 mysql mysql 969 Jun 10 14:44 cacert.pem -rw-r--r-- 1 mysql mysql 3078 Jun 10 14:44 mysql.crt -rw-r--r-- 1 mysql mysql 651 Jun 10 14:42 mysql.csr -rw------- 1 mysql mysql 887 Jun 10 14:40 mysql.key MariaDB [(none)]> show global variables like "%ssl%"; +---------------+-------------------------------+ | Variable_name | Value | +---------------+-------------------------------+ | have_openssl | YES | | have_ssl | YES | //已开启 | ssl_ca | /var/lib/mysql/ssl/cacert.pem | | ssl_capath | | | ssl_cert | /var/lib/mysql/ssl/mysql.crt | | ssl_cipher | | | ssl_key | /var/lib/mysql/ssl/mysql.key | +---------------+-------------------------------+ 7 rows in set (0.01 sec) MariaDB [(none)]> grant replication slave,replication client on *.* to ‘admin‘@‘172.18.250.78‘ identified by ‘admin‘ require ssl; //设置复制时需要基于SSL协议
从节点上：
]# vim /etc/my.cnf ssl //开启ssl ssl_ca=/var/lib/mysql/ssl/cacert.pem //ca证书 ssl_cert=/var/lib/mysql/ssl/slave.crt //从节点证书 ssl_key=/var/lib/mysql/ssl/slave.key //从节点私钥 ]# mkdir /var/lib/mysql/ssl ]# ll //根据openssl签署三个文件并放到这目录下 total 16 -rw-r--r--. 1 mysql mysql 969 Jun 10 14:44 cacert.pem -rw-r--r--. 1 mysql mysql 3077 Jun 10 16:39 slave.crt -rw-r--r--. 1 mysql mysql 651 Jun 10 16:38 slave.csr -rw-------. 1 mysql mysql 887 Jun 10 16:36 slave.key ]# service mariadb restart MariaDB [(none)]> grant all on *.* to ‘admin‘@‘127.0.0.1‘ identified by ‘admin‘ require ssl; Query OK, 0 rows affected (0.00 sec) //创建一个基于SSL通信的账号 ]# mysql -uadmin -h127.0.0.1 -padmin --ssl_ca=/var/lib/mysql/ssl/cacert.pem --ssl_cert=/var/lib/mysql/ssl/slave.crt --ssl_key=/var/lib/mysql/ssl/slave.key Welcome to the MariaDB monitor. Commands end with ; or \g. Your MariaDB connection id is 3 Server version: 5.5.44-MariaDB MariaDB Server Copyright (c) 2000, 2015, Oracle, MariaDB Corporation Ab and others. Type ‘help;‘ or ‘\h‘ for help. Type ‘\c‘ to clear the current input statement. MariaDB [(none)]> \s -------------- mysql Ver 15.1 Distrib 5.5.44-MariaDB, for Linux (x86_64) using readline 5.1 Connection id: 3 Current database: Current user: [email protected] SSL: Cipher in use is DHE-RSA-AES256-GCM-SHA384 //使用RSA算法加密
配置基于SSL的复制：
在从服务器上： MariaDB [(none)]> change master to master_host=‘172.18.250.77‘, master_user=‘admin‘, master_password=‘admin‘, master_log_file=‘master-bin.000010‘, master_log_pos=541, master_connect_retry=10, MASTER_HEARTBEAT_PERIOD=5, master_ssl=1, master_ssl_ca="/var/lib/mysql/ssl/cacert.pem", master_ssl_cert="/var/lib/mysql/ssl/slave.crt", master_ssl_key="/var/lib/mysql/ssl/slave.key"; MariaDB [(none)]> start slave; Query OK, 0 rows affected (0.01 sec) MariaDB [(none)]> show slave status\G; *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.18.250.77 Master_User: admin Master_Port: 3306 Connect_Retry: 10 Master_Log_File: master-bin.000010 Read_Master_Log_Pos: 541 Relay_Log_File: relay-log.000002 Relay_Log_Pos: 530 Relay_Master_Log_File: master-bin.000010 Slave_IO_Running: Yes Slave_SQL_Running: Yes Replicate_Do_DB: Replicate_Ignore_DB: Replicate_Do_Table: Replicate_Ignore_Table: Replicate_Wild_Do_Table: Replicate_Wild_Ignore_Table: Last_Errno: 0 Last_Error: Skip_Counter: 0 Exec_Master_Log_Pos: 541 Relay_Log_Space: 818 Until_Condition: None Until_Log_File: Until_Log_Pos: 0 Master_SSL_Allowed: Yes Master_SSL_CA_File: /var/lib/mysql/ssl/cacert.pem Master_SSL_CA_Path: Master_SSL_Cert: /var/lib/mysql/ssl/slave.crt Master_SSL_Cipher: Master_SSL_Key: /var/lib/mysql/ssl/slave.key Seconds_Behind_Master: 0
主服务器插入数据，从服务器是否能同步到：
MariaDB [(none)]> create database myssl; //主服务器上创建表 Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> show databases; //从服务器上查看 +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | myssl | //数据同步成功 | mytest | | performance_schema | | ssl | | test | +--------------------+ 7 rows in set (0.00 sec)
二、mysql的半同步复制模型
mysql的复制是异步实现的，mysql主服务器执行写操作后，把导致数据改变的数据同步到二进制日志中，从服务器只要读取里面新的二进制事件并在本地执行重放，mysql主服务器不必等待从服务器返回事务已提交完成的信息，直接 响应客户端数据已修改。
半同步复制时基于mysql的一主多从模式，通过在master端安装 semisync_master.so 插件，在slave端安装 semisync_slave.so 插件来实现半同步，这时主服务器需要从服务器返回事务已提交才返回给客户端数据已修改。如果从服务器一直不返回事务已提交的信息，那么则会降为异步复制（通过rpl_semi_sync_master_timeout 来设置）。
配置主服务器：
]# rpm -ql mariadb-server /usr/lib64/mysql/plugin/semisync_master.so //实现半同步复制的插件 /usr/lib64/mysql/plugin/semisync_slave.so MariaDB [(none)]> help install Name: ‘INSTALL PLUGIN‘ Description: Syntax: INSTALL PLUGIN plugin_name SONAME ‘shared_library_name‘ MariaDB [(none)]> grant replication slave,replication client on *.* to ‘admin‘@‘172.18.250.78‘ identified by ‘admin‘; //先授权复制账号 Query OK, 0 rows affected (10.01 sec) MariaDB [(none)]> install plugin rpl_semi_sync_master SONAME ‘semisync_master.so‘; Query OK, 0 rows affected (0.04 sec) //安装插件 MariaDB [(none)]> set global rpl_semi_sync_master_enabled=1; Query OK, 0 rows affected (0.00 sec) //启用插件 MariaDB [(none)]> show global variables like "%rpl_semi_%"; +------------------------------------+-------+ | Variable_name | Value | +------------------------------------+-------+ | rpl_semi_sync_master_enabled | ON | //开启半同步复制 | rpl_semi_sync_master_timeout | 10000 | //默认是10秒 | rpl_semi_sync_master_trace_level | 32 | //跟踪级别 | rpl_semi_sync_master_wait_no_slave | ON | //当没有从服务器时是否等待 +------------------------------------+-------+ 4 rows in set (0.01 sec) MariaDB [(none)]> show global status like "%rpl_semi%"; //查看半同步复制的状态信息 MariaDB [(none)]> show plugins; | rpl_semi_sync_master | ACTIVE | REPLICATION | semisync_master.so | GPL | +--------------------------------+----------+--------------------+--------------------+-- //插件已经激活
配置从节点：
MariaDB [(none)]> change master to master_host=‘172.18.250.77‘, master_user=‘admin‘, master_password=‘admin‘, master_log_file=‘master-bin.000003‘, master_log_pos=418; MariaDB [(none)]> install plugin rpl_semi_sync_slave soname "semisync_slave.so"; Query OK, 0 rows affected (0.13 sec) //安装插件 MariaDB [(none)]> set global rpl_semi_sync_slave_enabled=1; Query OK, 0 rows affected (0.00 sec) //启用插件 MariaDB [(none)]> show global variables like "%rpl_semi%"; +---------------------------------+-------+ | Variable_name | Value | +---------------------------------+-------+ | rpl_semi_sync_slave_enabled | ON | | rpl_semi_sync_slave_trace_level | 32 | +---------------------------------+-------+ 2 rows in set (0.01 sec) MariaDB [(none)]> start slave; Query OK, 0 rows affected (0.04 sec) MariaDB [(none)]> show slave status\G; *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.18.250.77 Master_User: admin Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000003 Read_Master_Log_Pos: 419 Relay_Log_File: relay-log.000002 Relay_Log_Pos: 530 Relay_Master_Log_File: master-bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes 在主服务器上创建数据库： MariaDB [(none)]> create database mytest; Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> show global status like "%rpl_semi%"; +--------------------------------------------+-------+ | Variable_name | Value | +--------------------------------------------+-------+ | Rpl_semi_sync_master_clients | 1 | //多少个半同步节点 | Rpl_semi_sync_master_net_avg_wait_time | 1367 | | Rpl_semi_sync_master_net_wait_time | 1367 | | Rpl_semi_sync_master_net_waits | 1 | | Rpl_semi_sync_master_no_times | 1 | | Rpl_semi_sync_master_no_tx | 1 | | Rpl_semi_sync_master_status | ON | | Rpl_semi_sync_master_timefunc_failures | 0 | | Rpl_semi_sync_master_tx_avg_wait_time | 1673 | | Rpl_semi_sync_master_tx_wait_time | 1673 | | Rpl_semi_sync_master_tx_waits | 1 | | Rpl_semi_sync_master_wait_pos_backtraverse | 0 | | Rpl_semi_sync_master_wait_sessions | 0 | | Rpl_semi_sync_master_yes_tx | 1 | +--------------------------------------------+-------+ 14 rows in set (0.01 sec) 从服务器上查看二进制日志是否重放： MariaDB [(none)]> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | mytest | //没问题 | performance_schema | | test | +--------------------+ 5 rows in set (0.01 sec)
三、mysql的主主复制模型
mysql主主复制其实就是两服务器互为主从，服务器既能读有能写，解决了单台主服务器下I/O压力过大的问题，两台服务器上都开启二进制日志和中继日志，只要有一方二进制发生改变就同步到本地中继日志进行重放。
配置一台服务器：
[mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock symbolic-links=0 skip_name_resolve=ON innodb_file_per_table=ON log-bin=master-bin //开启二进制日志 relay-log=relay-bin //开启中继日志 server_id=1 //唯一ID号，如果接受到的是自己ID号的二进制日志则不执行 auto_increment_offset=1 //设置自动增长字段为奇数字 auto_increment_increment=2 ]# service mariadb start Redirecting to /bin/systemctl start mariadb.service MariaDB [(none)]> show global variables like "%log%"; | log_bin | ON //二进制日志开启 | relay_log | relay-bin //中继日志开启 MariaDB [(none)]> show global variables like "%increment%"; +-----------------------------+-------+ | Variable_name | Value | +-----------------------------+-------+ | auto_increment_increment | 2 | | auto_increment_offset | 1 | | div_precision_increment | 4 | | innodb_autoextend_increment | 8 | +-----------------------------+-------+ 4 rows in set (0.00 sec) MariaDB [(none)]> grant replication slave, replication client on *.* to ‘admin1‘@‘172.18.250.78‘ identified by ‘admin1‘; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> change master to master_host=‘172.18.250.78‘, master_user=‘admin2‘, master_password=‘admin2‘, master_log_file="master-bin.000003", master_log_pos=607; Query OK, 0 rows affected (0.01 sec) 在另一台上授权： MariaDB [(none)]> grant replication slave, replication client on *.* to ‘admin2‘@‘172.18.250.77‘ identified by ‘admin2‘; Query OK, 0 rows affected (0.00 sec) MariaDB [(none)]> change master to master_host=‘172.18.250.77‘, master_user=‘admin1‘, master_password=‘admin1‘, master_log_file="master-bin.000003", master_log_pos=426; Query OK, 0 rows affected (0.08 sec) 都启动slave复制线程： MariaDB [(none)]> start slave; Query OK, 0 rows affected (0.01 sec) MariaDB [(none)]> show slave status\G; //在250.78上查看 *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.18.250.77 Master_User: admin1 Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000003 Read_Master_Log_Pos: 426 Relay_Log_File: relay-bin.000002 Relay_Log_Pos: 530 Relay_Master_Log_File: master-bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes MariaDB [(none)]> show slave status\G; //在250.77上查看 *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event Master_Host: 172.18.250.78 Master_User: admin2 Master_Port: 3306 Connect_Retry: 60 Master_Log_File: master-bin.000003 Read_Master_Log_Pos: 607 Relay_Log_File: relay-bin.000002 Relay_Log_Pos: 530 Relay_Master_Log_File: master-bin.000003 Slave_IO_Running: Yes Slave_SQL_Running: Yes
可以看到有三个线程在运行，一个是dump线程，一个是I/O线程，还有一个SQL线程。
测试是否能同步：
MariaDB [(none)]> create database hellodb; //250.77上创建数据库 Query OK, 1 row affected (0.00 sec) MariaDB [(none)]> show databases; //250.78上查看数据库 +--------------------+ | Database | +--------------------+ | information_schema | | hellodb | | mysql | | performance_schema | | test | +--------------------+ 5 rows in set (0.00 sec) MariaDB [hellodb]> create table students(id int unsigned not null primary key,name char(30) not null,age tinyint unsigned,gender enum(‘f‘,‘m‘)); //250.78上创建表 Query OK, 0 rows affected (0.18 sec) MariaDB [hellodb]> desc students; //250.77上查看表 +--------+---------------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +--------+---------------------+------+-----+---------+-------+ | id | int(10) unsigned | NO | PRI | NULL | | | name | char(30) | NO | | NULL | | | age | tinyint(3) unsigned | YES | | NULL | | | gender | enum(‘f‘,‘m‘) | YES | | NULL | | +--------+---------------------+------+-----+---------+-------+ 4 rows in set (0.15 sec)
以上是关于mysql的主从主主及半同步复制的主要内容，如果未能解决你的问题，请参考以下文章