pt-online-schema-change工作原理

Posted 2020-06-17

./pt-online-schema-change  --user=root --password=root456 --recursion-method="processlist" --chunk-size=50000 --alter="modify column emp_no bigint not null auto_increment"   h=192.168.31.64,P=3306,D=employees,t=tmp_employees --execute 

创建一张和原表结构一样的中间表，命名一般都是_开头，new结尾

CREATE TABLE `employees`.`_tmp_employees_new` (

  `emp_no` bigint(19) NOT NULL AUTO_INCREMENT,

  `birth_date` date NOT NULL,

  `first_name` varchar(20) NOT NULL,

  `last_name` varchar(16) NOT NULL,

  `gender` enum(‘M‘,‘F‘) NOT NULL,

  `hire_date` date NOT NULL,

  PRIMARY KEY (`emp_no`),

  KEY `idx_1` (`hire_date`)

) ENGINE=InnoDB AUTO_INCREMENT=500000 DEFAULT CHARSET=utf8;

将中间表按条件修改表结构

ALTER TABLE `employees`.`_tmp_employees_new` modify column emp_no bigint not null auto_increment;

创建触发器，当在拷贝过程中发现原表有增删改操作，那么由触发器同步到临时创建的表中

                   93 Query     CREATE TRIGGER `pt_osc_employees_tmp_employees_del` AFTER DELETE ON `employees`.`tmp_employees` FOR EACH ROW DELETE IGNORE FROM `employees`.`_tmp_employees_new` WHERE `employees`.`_tmp_employees_new`.`emp_no` <=> OLD.`emp_no`

                   93 Query     CREATE TRIGGER `pt_osc_employees_tmp_employees_upd` AFTER UPDATE ON `employees`.`tmp_employees` FOR EACH ROW REPLACE INTO `employees`.`_tmp_employees_new` (`emp_no`, `birth_date`, `first_name`, `last_name`, `gender`, `hire_date`) VALUES (NEW.`emp_no`, NEW.`birth_date`, NEW.`first_name`, NEW.`last_name`, NEW.`gender`, NEW.`hire_date`)

                   93 Query     CREATE TRIGGER `pt_osc_employees_tmp_employees_ins` AFTER INSERT ON `employees`.`tmp_employees` FOR EACH ROW REPLACE INTO `employees`.`_tmp_employees_new` (`emp_no`, `birth_date`, `first_name`, `last_name`, `gender`, `hire_date`) VALUES (NEW.`emp_no`, NEW.`birth_date`, NEW.`first_name`, NEW.`last_name`, NEW.`gender`, NEW.`hire_date`)

                   93 Query     EXPLAIN SELECT * FROM `employees`.`tmp_employees` WHERE 1=1

进行数据拷贝，默认的chunk-size值是1000，即每次拷贝1000条数据，拷贝过程中需要加个s锁，那么此时是不可以做修改操作的，但是这种拷贝一般会非常快，影响很小，由于需要加锁，所以你的chunk-size值不要指定的过大，本文中是开启了general-log，表数据总共有30万条，为了让少打日志，所以指定了50000

 INSERT LOW_PRIORITY IGNORE INTO `employees`.`_tmp_employees_new` (`emp_no`, `birth_date`, `first_name`, `last_name`, `gender`, `hire_date`) SELECT `emp_no`, `birth_date`, `first_name`, `last_name`, `gender`, `hire_date` FROM `employees`.`tmp_employees` FORCE INDEX(`PRIMARY`) WHERE ((`emp_no` >= ‘10001‘)) AND ((`emp_no` <= ‘60000‘)) LOCK IN SHARE MODE /*pt-online-schema-change 15147 copy nibble*/

。。。。。。。。。

 INSERT LOW_PRIORITY IGNORE INTO `employees`.`_tmp_employees_new` (`emp_no`, `birth_date`, `first_name`, `last_name`, `gender`, `hire_date`) SELECT `emp_no`, `birth_date`, `first_name`, `last_name`, `gender`, `hire_date` FROM `employees`.`tmp_employees` FORCE INDEX(`PRIMARY`) WHERE ((`emp_no` >= ‘499976‘)) AND ((`emp_no` <= ‘499999‘)) LOCK IN SHARE MODE /*pt-online-schema-change 15147 copy nibble*/

给中间表做表统计

 ANALYZE TABLE `employees`.`_tmp_employees_new` /* pt-online-schema-change */

将原表命名为_tmp_employees_old，将中间表命名为原表

RENAME TABLE `employees`.`tmp_employees` TO `employees`.`_tmp_employees_old`, `employees`.`_tmp_employees_new` TO `employees`.`tmp_employees`

删除old表

DROP TABLE IF EXISTS `employees`.`_tmp_employees_old`

删除第三步创建的触发器

                   95 Query     DROP TABLE IF EXISTS `employees`.`_tmp_employees_old`

                   95 Query     DROP TRIGGER IF EXISTS `employees`.`pt_osc_employees_tmp_employees_del`

                   95 Query     DROP TRIGGER IF EXISTS `employees`.`pt_osc_employees_tmp_employees_upd`

                   95 Query     DROP TRIGGER IF EXISTS `employees`.`pt_osc_employees_tmp_employees_ins`