尝试OUTFIleINFILE快速导入导出数据

Posted 2020-12-29

尝试OUTFIle、INFILE快速导入导出数据

应用场景：

前几天开发突然有这么一个需求，想导一份200多G的mysql数据出来到另一台机器上，而且时间有点赶，第一时间就想要使用Xtrabackup来全备与增备。但想到之前使用Xtrabackup来备份恢复的时候出现了各种坑，就问了下同事有什么好建议来快速导出导入数据，后来知道了可以使用select into outfile导出表数据，就冒着尝试一下的心里去弄了一下，得到的结果是惊人的，个人感觉速度要比Xtrabackup快很多。

使用select into outfile导出表数据：

（一个for循环定义自己需要操作的数据库名称，把数据导入到/data/tmp目录下）

for table in `echo oat_inventory_in oat_inventory_out oat_inventory_defective_out oat_reject oat_reject_line oat_goods oat_goods_related oat_order oat_order_line oat_purchase oat_purchase_line oat_invcheck oat_inventory_deal oat_stage_invent oat_inventory oat_receives oat_withdraw oat_deduct oat_order_provide_amount oat_order_coordinate oat_order_distribution oat_refund oat_entity_amount oat_stage_entityfund oat_entity_frozen_detail oat_entity oat_funds_detail`
do
echo $table
mysql -u root -pPassword dbname -e "select * into outfile ‘/data/tmp/$table.txt‘ fields terminated by ‘,‘ from $table;"
done

导出表结构：

（因为上述只是倒入数据，而表的结构则需要使用mysqldump方式去导出）

/usr/local/mysql/bin/mysqldump -u root -pPassword -d dbname oat_inventory_in oat_inventory_out oat_inventory_defective_out oat_reject oat_reject_line oat_goods oat_goods_related oat_order oat_order_line oat_purchase oat_purchase_line oat_invcheck oat_inventory_deal oat_stage_invent oat_inventory oat_receives oat_withdraw oat_deduct oat_order_provide_amount oat_order_coordinate oat_order_distribution oat_refund oat_entity_amount oat_stage_entityfund oat_entity_frozen_detail oat_entity oat_funds_detail > struct.sql

将导出的结构与数据文件scp到目标主机上（建议数据scp之前先压缩）：

scp -P 22 /data/tmp/*.gz [email protected]:/data 

在新的数据库上面导入表结构：

mysql -u root -pPassword dbname <  struct.sql

使用 load data infile 导入数据：

for table in `echo oat_inventory_out oat_inventory_defective_out oat_reject oat_reject_line oat_goods oat_goods_related oat_order oat_order_line oat_purchase oat_purchase_line oat_invcheck oat_inventory_deal oat_stage_invent oat_inventory oat_receives oat_withdraw oat_deduct oat_order_provide_amount oat_order_coordinate oat_order_distribution oat_refund oat_entity_amount oat_stage_entityfund oat_entity_frozen_detail oat_entity oat_funds_detail`
do
echo $table
mysql -u root -pPassword dbname -e "LOAD DATA INFILE ‘/home/tmp/$table.txt‘ INTO TABLE $table FIELDS TERMINATED BY ‘,‘"
done