单台设备基于63G的数据量快速完成mysql主从搭建

Posted 2020-11-25

一、演示课题说明：

单台物理机利用xtrabackup工具在线备份63G的mysql数据，来新建slave库。
演示的目的主要是记录下在单台物理服务器上利用63G的测试库数据，然后在本机上快速新建一个slave库，大概需要多久完成。以及在新增的slave的过程中对master库锁表影响多大？

二、设备和系统环境说明:

设备环境： x86_64位最小化安装

 [root@localhost scripts]# cat /etc/redhat-release 
CentOS Linux release 7.5.1804 (Core) 

设备和硬盘型号：
双硬盘：SSD盘-intel 单盘raid0

+--------------------------------------------------------------+
| This Machine‘s Hyper-Threading is Enabled(recommend disable) |
+--------------------------------------------------------------+
Systembit    :    64
MEM info     :    6*16384 MB
Disk_totle   :    
Pro_SN_name  :    Product Name: PowerEdge R630 Serial Number: G8WFKM2
System name  :    S
Board_SN name:    Product Name: 02C2CP Serial Number: .G8WFKM2.CNIVC0079R1641.
CPU model    :    48  Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz
CPU_phy_num  :    2
CPU_core_num :    12
CPU_logic_num:    24
Netcard info :    Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe
+++++++++++++++++++++++++++++++++++
<<raid informations>>
 Virtual Drive: 0 (Target Id: 0)
Name                :
RAID Level          : Primary-0, Secondary-0, RAID Level Qualifier-0
Size                : 223.0 GB
State               : Optimal
Strip Size          : 64 KB
Number Of Drives    : 1
--
Virtual Drive Information:
Virtual Drive: 1 (Target Id: 1)
Name                :
RAID Level          : Primary-0, Secondary-0, RAID Level Qualifier-0
Size                : 223.0 GB
State               : Optimal
Strip Size          : 64 KB
Number Of Drives    : 1
+++++++++++++++++++++++++++++++++++
<<Virtual Drive Information>>
Device Id: 0
PD Type: SATA
Raw Size: 223.570 GB [0x1bf244b0 Sectors]
Inquiry Data: PHDV7234023C240AGN  INTEL SSDSC2BB240G7                     N2010112
Device Speed: 6.0Gb/s 
Drive Temperature :27C (80.60 F)
Device Id: 1
PD Type: SATA
Raw Size: 223.570 GB [0x1bf244b0 Sectors]
Inquiry Data: PHDV717003D9240AGN  INTEL SSD
Device Speed: 6.0Gb/s 
Drive Temperature :26C (78.80 F)

三、创建测试数据：

创建测试表：

 CREATE TABLE `test_event` (
`id` int(8) NOT NULL AUTO_INCREMENT, 
`username` varchar(20) COLLATE utf8_unicode_ci NOT NULL,
`password` varchar(20) COLLATE utf8_unicode_ci NOT NULL, 
`create_time` varchar(20) COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`id`) #主键ID
) ENGINE=innodb AUTO_INCREMENT=0 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;

创建定时器，每一秒写入数据：

delimiter $$
create event event_2 
on schedule every 1 second STARTS   CURRENT_TIMESTAMP + INTERVAL 1 MINUTE
COMMENT ‘xiaowu create‘
do 
    BEGIN
           insert into test_event(username,password,create_time) values("李四","tomcat",now());
    END $$
delimiter ;

提示：
创建上诉的测试数据，主要是想看下在xtrabackup备份数据时，会 对master库锁表多长时间，以及锁表是否会导致insert数据失败
此次演示中虽然每一秒都有写入数据到master库中，但是并没有看到锁表造成写入的表数据失败（可能还是每秒并发写入表的数据太少，或者是锁表的过程时间太短）

四、备份恢复数据并搭建主从复制

xtrabackup备份工具安装过程此处不再赘述，具体可以看考此链接：
https://blog.51cto.com/wujianwei/2430281

4.1xtrackup备份数据：

time innobackupex --defaults-file=/etc/my.cnf  -ubackupuser -p654321ccs  --host=127.0.0.1  -S /tmp/mysql.sock --parallel=2 --no-timestamp /data/backup/db_3306_`date +%Y%m%d`

200115 17:22:33 Executing UNLOCK TABLES
200115 17:22:33 All tables unlocked
200115 17:22:33 [00] Copying ib_buffer_pool to /data/backup/db_3306_20200115/ib_buffer_pool
200115 17:22:33 [00]        ...done
200115 17:22:33 Backup created in directory ‘/data/backup/db_3306_20200115/‘
MySQL binlog position: filename ‘mysql-bin.000109‘, position ‘491959802‘, GTID of the last change ‘64a062d1-2e92-11ea-847e-801844ed7bbc:1-84402‘
200115 17:22:33 [00] Writing /data/backup/db_3306_20200115/backup-my.cnf
200115 17:22:33 [00]        ...done
200115 17:22:33 [00] Writing /data/backup/db_3306_20200115/xtrabackup_info
200115 17:22:33 [00]        ...done
xtrabackup: Transaction log of lsn (117743624292) to (117743935718) was copied.
200115 17:22:33 completed OK!

real    11m21.125s
user    0m49.376s
sys 2m15.140s

耗时11分钟21秒

[root@localhost backup]# time innobackupex  --apply-log  /data/backup/db_3306_20200115/
InnoDB: Shutdown completed; log sequence number 117743937576
200115 17:38:41 completed OK!

real    0m14.873s
user    0m0.093s
sys 0m2.428s

耗时15秒

4.2恢复备份的数据到3307新实例：

[root@localhost 3307]# time innobackupex --defaults-file=/data1/mysql/3307/my3307.cnf  --parallel=2 --use-memory=1024M --copy-back /data/backup/db_3306_20200115/
200115 17:49:35 [01] Copying ./ibtmp1 to /data1/mysql/3307/data/ibtmp1
200115 17:49:35 [01]        ...done
200115 17:49:59 [02]        ...done
200115 17:49:59 completed OK!

real    6m25.941s
user    0m0.122s
sys 3m21.198s

耗时6分26秒

4.3授权3307新实例data目录mysql的权限：

chown -R mysql.mysql /data1/mysql/3307/data 

4.4启动3307 mysql服务

4.5 master库上创建复制用户:

grant replication slave on *.* to rep@‘127.0.0.1‘ identified by ‘JuwoSdk21TbUser‘; flush privileges;

查看xtrabackup 备份的完binlog的位置点：

cat /data/backup/db_3306_20200115/xtrabackup_binlog_info
[root@localhost db_3306_20200115]# cat /data/backup/db_3306_20200115/xtrabackup_binlog_info
mysql-bin.000109    491959802   64a062d1-2e92-11ea-847e-801844ed7bbc:1-84402

4.6 采用gtid和不采用Gtid 两种方式配置复制：

slave库上change master to:

change master to master_host=‘127.0.0.1‘,master_user=‘rep‘,master_password=‘JuwoSdk21TbUser‘,master_log_file=‘mysql-bin.000109‘,master_log_pos=491959802;start slave;show slave statusG

采用上诉的方式配置主从复制，说明master库一开始没开启Gtid。
+++++++++++++++
如果master库上一开始就开启Gtid，则在配置主从复制时slave库采用下面的命令：

  CHANGE MASTER TO
  MASTER_HOST=‘127.0.0.1‘,
  MASTER_USER=‘rep‘,
  MASTER_PASSWORD=‘JuwoSdk21TbUser‘,
  MASTER_PORT=3306,
  MASTER_AUTO_POSITION = 1;

注意：Gtid主从复制时，master库事先一定是开启Gtid的

在恢复xttabackup备份的数据到新实例上并启动新实例后，要登录这个新的实例，清空下新实例上的gtid的信息，
清除新的3307实例的Gtid信息：

mysql> reset master;

查看xtrabackup 备份的完binlog的位置点：

cat /data/backup/db_3306_20200115/xtrabackup_binlog_info
[root@localhost db_3306_20200115]# cat /data/backup/db_3306_20200115/xtrabackup_binlog_info
mysql-bin.000109    491959802   64a062d1-2e92-11ea-847e-801844ed7bbc:1-84402

然后再执行如下命令：

  CHANGE MASTER TO
  MASTER_HOST=‘127.0.0.1‘,
  MASTER_USER=‘rep‘,
  MASTER_PASSWORD=‘JuwoSdk21TbUser‘,
  MASTER_PORT=3306,
  MASTER_AUTO_POSITION = 1;

总结：整个备份，恢复和新建slave，基于63G的数据量在18分钟就可以完成。而且xtrabackup 备份过程中锁表和解锁在1s内就完成了，在模拟的环境中看，锁表的影响还是很小的。但是还是建议在业务低峰期进行xtrabackup 备份数据以及在slave库上进行 xtrabckup备份数据，将锁表对数据库的影响降低到最小化