xtrabackup模拟备份100G数据

Posted 2022-07-13

一、环境说明

dell r620 双颗CPU,单核心是12核心，内存是72G ,硬盘伪sata接口的企业级的SSD固态硬盘：500G

固态盘型号如下：

PD Type: SATA
Raw Size: 465.761 GB [0x3a386030 Sectors]
Inquiry Data: S3Z3NB0K116201W     Samsung SSD 860 EVO 500GB               RVT01B6Q
Device Speed: 6.0Gb/s 

二、测试项目

测试项目：使用xtrabackup 备份100G的数据使用的时间

xtrabckup备份工具安装此处忽略

授权xtrabackup 备份数据时登录库的用户：

grant reload,lock tables,process,replication client,super on *.* to backupuser@‘127.0.0.1‘ identified by ‘654321ccs‘;flush privileges;

测试使用xtrabackup 单线程备份100G的数据使用的时间是17分39秒

备份命令：

[root@localhost ~]# time innobackupex --defaults-file=/etc/my.cnf  -ubackupuser -p654321ccs  --host=127.0.0.1  -S /tmp/mysql3306.sock  --no-timestamp 

190812 16:40:52 Finished backing up non-InnoDB tables and files
190812 16:40:52 [00] Writing /data/backup/db_3306_20190812/xtrabackup_binlog_info
190812 16:40:52 [00]        ...done
190812 16:40:52 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS...
xtrabackup: The latest check point (for incremental): ‘143003390816‘
xtrabackup: Stopping log copying thread.
.190812 16:40:52 >> log scanned up to (143003390816)

190812 16:40:52 Executing UNLOCK TABLES
190812 16:40:52 All tables unlocked
190812 16:40:52 [00] Copying ib_buffer_pool to /data/backup/db_3306_20190812/ib_buffer_pool
190812 16:40:52 [00]        ...done
190812 16:40:52 Backup created in directory ‘/data/backup/db_3306_20190812/‘
MySQL binlog position: filename ‘mysql-bin.000060‘, position ‘324009766‘
190812 16:40:52 [00] Writing /data/backup/db_3306_20190812/backup-my.cnf
190812 16:40:52 [00]        ...done
190812 16:40:52 [00] Writing /data/backup/db_3306_20190812/xtrabackup_info
190812 16:40:52 [00]        ...done
xtrabackup: Transaction log of lsn (143003390806) to (143003390816) was copied.
190812 16:40:52 completed OK!

real    17m39.710s
user    6m23.436s
sys     2m19.513s

从zabbix上查看监控磁盘io的使用率如下：

**从zabbix上查看监控磁盘io的使用率刚超过设定的阈值50%，不到60%，同时每秒的磁盘写入量如下：

也就是此时采用3个线程并行备份数据刚刚好。磁盘的io繁忙度可以接受

测试使用xtrabackup 开启3个线程备份100G的数据使用的时间是9分44秒：

time innobackupex --defaults-file=/etc/my.cnf  -ubackupuser -p123456ccs  --host=127.0.0.1  -S /tmp/mysql3306.sock  --parallel=3 --no-timestamp /data/backup/db_3306_`date +%Y%m%d`

190812 16:55:07 Finished backing up non-InnoDB tables and files
190812 16:55:07 [00] Writing /data/backup/db_3306_20190812/xtrabackup_binlog_info
190812 16:55:07 [00]        ...done
190812 16:55:07 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS...
xtrabackup: The latest check point (for incremental): ‘143003390816‘
xtrabackup: Stopping log copying thread.
.190812 16:55:07 >> log scanned up to (143003390816)

190812 16:55:07 Executing UNLOCK TABLES
190812 16:55:07 All tables unlocked
190812 16:55:07 [00] Copying ib_buffer_pool to /data/backup/db_3306_20190812/ib_buffer_pool
190812 16:55:07 [00]        ...done
190812 16:55:07 Backup created in directory ‘/data/backup/db_3306_20190812/‘
MySQL binlog position: filename ‘mysql-bin.000060‘, position ‘324017447‘
190812 16:55:07 [00] Writing /data/backup/db_3306_20190812/backup-my.cnf
190812 16:55:07 [00]        ...done
190812 16:55:07 [00] Writing /data/backup/db_3306_20190812/xtrabackup_info
190812 16:55:07 [00]        ...done
xtrabackup: Transaction log of lsn (143003390816) to (143003390816) was copied.
190812 16:55:07 completed OK!

real    9m44.015s
user    5m26.039s
sys     2m26.201s

说明在使用3个线程备份数据时，已经是达到了SSD固态盘io写入的极限。不可能再快了

同时备份期间查看iostat命令查看SSD磁盘 设备：sdb 的已经是100%繁忙了。达到了磁盘写入性能的极限。

[root@localhost backup]# iostat -d -x -k 2 30
Linux 3.10.0-327.el7.x86_64 (localhost.localdomain)     08/12/2019  _x86_64_    (32 CPU)

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdb               0.00     0.57    0.34    3.26    42.53   256.11   166.31     0.05   13.46   38.31   10.90   0.26   0.09
sda               0.00     0.27    0.00    1.35     0.06     9.31    13.84     0.00    1.54    0.51    1.54   0.13   0.02

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdb               0.00     1.50 1040.00 1674.50 132442.00 213625.50   254.98   224.57   83.06   46.46  105.79   0.37 100.00
sda               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00
。。。。。。。。。。。。。。。。
Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdb               0.00     1.00 1294.00 1417.00 164846.00 180440.75   254.73   202.51   74.50   43.77  102.56   0.37 100.00
sda               0.00     0.00    0.00    4.00     0.00    16.25     8.12     0.00    0.75    0.00    0.75   0.75   0.30

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await r_await w_await  svctm  %util
sdb               0.00     2.00 1616.50 1366.00 206328.00 174046.25   255.07   223.71   75.18   47.35  108.12   0.34 100.00
sda               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00    0.00    0.00   0.00   0.00

从zabbix上查看监控磁盘io的使用率已经是100%，磁盘的写入和读取基本都在185M/s

这个已经是达到次SSD盘的io性能的极限了。

测试使用xtrabackup 开启4个线程备份100G的数据使用的时间是9分58秒：

time innobackupex --defaults-file=/etc/my.cnf  -ubackupuser -p123456ccs  --host=127.0.0.1  -S /tmp/mysql3306.sock  --parallel=4 --no-timestamp /data/backup/db_3306_`date +%Y%m%d`

190812 17:11:55 Finished backing up non-InnoDB tables and files
190812 17:11:55 [00] Writing /data/backup/db_3306_20190812/xtrabackup_binlog_info
190812 17:11:55 [00]        ...done
190812 17:11:55 Executing FLUSH NO_WRITE_TO_BINLOG ENGINE LOGS...
xtrabackup: The latest check point (for incremental): ‘143003390816‘
xtrabackup: Stopping log copying thread.
.190812 17:11:55 >> log scanned up to (143003390816)

190812 17:11:55 Executing UNLOCK TABLES
190812 17:11:55 All tables unlocked
190812 17:11:55 [00] Copying ib_buffer_pool to /data/backup/db_3306_20190812/ib_buffer_pool
190812 17:11:55 [00]        ...done
190812 17:11:55 Backup created in directory ‘/data/backup/db_3306_20190812/‘
MySQL binlog position: filename ‘mysql-bin.000060‘, position ‘324048095‘
190812 17:11:55 [00] Writing /data/backup/db_3306_20190812/backup-my.cnf
190812 17:11:55 [00]        ...done
190812 17:11:55 [00] Writing /data/backup/db_3306_20190812/xtrabackup_info
190812 17:11:55 [00]        ...done
xtrabackup: Transaction log of lsn (143003390816) to (143003390816) was copied.
190812 17:11:55 completed OK!

real    9m58.171s
user    5m12.634s
sys     2m43.506s