select into outfile 与 load data infile

Posted ywxbbbbb

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了select into outfile 与 load data infile相关的知识,希望对你有一定的参考价值。

select into outfile用法

mysql中,可以使用SELECT...INTO OUTFILE语句将表的内容导出为一个文本文件。

SELECT [列名] FROM table [WHERE 语句]
        INTO OUTFILE ‘目标文件‘ [OPTION];

  

“OPTION”参数为可选参数选项,其可能的取值有:

  • FIELDS TERMINATED BY ‘字符串‘:设置字符串为字段之间的分隔符,可以为单个或多个字符。默认值是“ ”。
  • FIELDS ENCLOSED BY ‘字符‘:设置字符来括住字段的值,只能为单个字符。默认情况下不使用任何符号。
  • FIELDS OPTIONALLY ENCLOSED BY ‘字符‘:设置字符来括住CHAR、VARCHAR和TEXT等字符型字段。默认情况下不使用任何符号。
  • FIELDS ESCAPED BY ‘字符‘:设置转义字符,只能为单个字符。默认值为“”。
  • LINES STARTING BY ‘字符串‘:设置每行数据开头的字符,可以为单个或多个字符。默认情况下不使用任何字符。
  • LINES TERMINATED BY ‘字符串‘:设置每行数据结尾的字符,可以为单个或多个字符。默认值是“ ”。

举个栗子:

select * from raptor.loan where DATE_FORMAT(create_at,‘%Y-%m-%d‘)="2018-10-21" or DATE_FORMAT(update_at,‘%Y-%m-%d‘)="2018-10-21" or repaired=1 into outfile "/tmp/raptor_loan_incre_2018-10-21.csv" FIELDS TERMINATED BY ‘|‘ LINES TERMINATED BY ‘ ‘;

将raptor.loan表2018-10-21日的增量数据导出到excel表,字符串为字段之间的分隔符 | ,每行数据结尾的字符 回车

load data infile用法

可以将select into outfile 导出的文本文件导入数据库

LOAD DATA INFILE "/path/to/file" INTO TABLE table_name;
注意:如果导出时用到了FIELDS TERMINATED BY ‘,‘ OPTIONALLY ENCLOSED BY ‘"‘ LINES TERMINATED BY ‘
‘语句,那么LOAD时也要加上同样的分隔限制语句。还要注意编码问题。

举个栗子:

LOAD DATA INFILE ‘/tmp/raptor_loan_track_incre_2018-10-21.csv‘ REPLACE INTO TABLE raptor.loan_track FIELDS TERMINATED BY ‘|‘ LINES TERMINATED BY ‘ ‘;

将文件/tmp/raptor_loan_track_incre_2018-10-21.csv导入raptor.loan_track表

 

报错

1、在操作导入的时候发现会出现主键冲突报错,原因是有两张表是全量备份的,再导入时可以使用参数REPLACE,MySQL会把相同的先干掉,再插入新的值。

 

2、导入时报错

技术分享图片

 

原因是因为excel表中存在空值,无法导入

 

数据库差量数据备份脚本

#!/bin/bash
date2=$(date -d yesterday +%Y-%m-%d)
sql1="select * from raptor.loan where DATE_FORMAT(create_at,‘%Y-%m-%d‘)="${date2}" or DATE_FORMAT(update_at,‘%Y-%m-%d‘)="${date2}" or repaired=1 into outfile "/tmp/raptor_loan_incre_${date2}.csv" FIELDS TERMINATED BY ‘|‘ LINES TERMINATED BY ‘
‘;"
sql2="select * from raptor.loan_track where DATE_FORMAT(create_at,‘%Y-%m-%d‘)="${date2}" or DATE_FORMAT(update_at,‘%Y-%m-%d‘)="${date2}" or repaired=1 into outfile "/tmp/raptor_loan_track_incre_${date2}.csv" FIELDS TERMINATED BY ‘|‘ LINES TERMINATED BY ‘
‘;"
sql3="select * from loan_extend where DATE_FORMAT(create_at,‘%Y-%m-%d‘)="${date2}" into outfile "/tmp/raptor_loan_extend_incre_${date2}.csv" FIELDS TERMINATED BY ‘|‘ LINES TERMINATED BY ‘
‘;"
sql4="select * from tenants into outfile "/tmp/raptor_tenants_incre_${date2}.csv" FIELDS TERMINATED BY ‘|‘ LINES TERMINATED BY ‘
‘;"
sql5="select * from user_privilege into outfile "/tmp/raptor_user_privilege_incre_${date2}.csv" FIELDS TERMINATED BY ‘|‘ LINES TERMINATED BY ‘
‘;"
sql6="select * from users into outfile "/tmp/raptor_users_incre_${date2}.csv" FIELDS TERMINATED BY ‘|‘ LINES TERMINATED BY ‘
‘;"
sql7="update loan_track set repaired=0 where repaired=1;"
sql8="update loan set repaired=0 where repaired=1;"
#sql9="select count(1) from raptor.loan where DATE_FORMAT(create_at,‘%Y-%m-%d‘)="${date2}" or DATE_FORMAT(update_at,‘%Y-%m-%d‘)="${date2}" or repaired=1"
execute_sql(){
	/usr/local/mysql/bin/mysql -u xxxxxxxxx -p‘xxxxxxxxxxxxxxxx‘  $DB -e "${1}"
	if [ -f /tmp/${2}_incre_${date2}.csv ]
		then 
			scp -P xxxxx /tmp/${2}_incre_${date2}.csv [email protected]:/data/sftp_docker/hadoop_ftp/raptor_repair/
			if [ $? == 0 ]
				then
					cd /tmp/
					rm ${2}_incre_${date2}.csv
			fi
	fi
}
/usr/local/mysql/bin/mysql -u xxxxxxxx -p‘xxxxxxxxxxxxxxx‘  raptor -e "${sql9}" > /root/sh/count.log
execute_sql "${sql1}" "raptor_loan"
execute_sql "${sql2}" "raptor_loan_track"
execute_sql "${sql3}" "raptor_loan_extend"
execute_sql "${sql4}" "raptor_tenants"
execute_sql "${sql5}" "raptor_user_privilege"
execute_sql "${sql6}" "raptor_users"
execute_sql "${sql7}"
execute_sql "${sql8}"

  

 

以上是关于select into outfile 与 load data infile的主要内容,如果未能解决你的问题,请参考以下文章

select into outfile

翻译:select into outfile(已提交到MariaDB官方手册)

select into outfile

MySQL SELECT xxx INTO OUTFILE用法

SELECT INTO OUTFILE 无法写入文件

Mysql 使用 select into outfile