sqoop 测试 --hive-delims-replacement 参数

Posted 2020-10-10 牵牛花

在hive的官方文档中给的例子中字段分隔符是\001,但是在他的API文档中--hive-delims-replacement ，--hive-drop-import-delims   参数中会处理的字符是\0x01,一个时8进制的1，一个是16进制的1，有这么一点差别，lz为了确认这两个是否一样故做了下面的实验

 

Example:

CREATE TABLE page_view(viewTime INT, userid BIGINT,      page_url STRING, referrer_url STRING,      ip STRING COMMENT ‘IP Address of the User‘)  COMMENT ‘This is the page view table‘  PARTITIONED BY(dt STRING, country STRING)  ROW FORMAT DELIMITED    FIELDS TERMINATED BY ‘\001‘ STORED AS SEQUENCEFILE;

The above statement lets you create the same table as the previous table.

In the previous examples the data is stored in <hive.metastore.warehouse.dir>/page_view. Specify a value for the key hive.metastore.warehouse.dir in the Hive config file hive-site.xml.

--hive-delims-replacement <arg>             Replace Hive record \0x01

                                               and row delimiters (\n\r)

                                               from imported string fields

                                               with user-defined string

--hive-drop-import-delims                   Drop Hive record \0x01 and

                                               row delimiters (\n\r) from

                                               imported string fields

这两个参数不能一起用

元数据（中间的方格是\001,用java代码生成的）

11 QQ
jyyh qwqwqw 1 1111 2017/10/15 23:27:48
15 java
jyyh 中文 2 1212 2017/10/15 23:39:57

sqoop import --connect jdbc:oracle:thin:@MSI:1521/study --username luo --password Sys_20170929 --table TB_NEWS --fields-terminated-by "\001" --lines-terminated-by "\n" --hive-import --hive-overwrite --null-string "" --null-non-string "" --fetch-size 1000 -m 3 --create-hive-table  --hive-table luoqi_test.TB_NEWS --delete-target-dir 

导入之后hive的结构是

11.0 QQ|jyyh qwqwqw 1 1111 2017-10-15 23:27:48.0
15.0 java|jyyh 中文 2 1212 2017-10-15 23:39:57.0

结果，‘\001‘被替换成正确的字符

结论，\001可以被--hive-delims-replacement 参数替换

 

导出将这个符号还原？