我的 CSV 文件带有双引号括起来的字段 - 无法识别数值“12131”

Posted 2023-03-29

【中文标题】我的 CSV 文件带有双引号括起来的字段 - 无法识别数值“12131”

【英文标题】：My CSV file with double quotes enclosed fields - numeric value ' "12131" ' not recognized

【发布时间】：2019-12-18 21:05:31

【问题描述】：

我暂存了一个 csv 文件，所有字段都用双引号 (" ") 和逗号分隔，行由换行符分隔。封闭字段中的值还包含换行符 (\n)。

我使用默认的文件格式 = CSV。使用 COPY INTO 时，我在这种情况下看到列不匹配错误。

我通过在下面的 SQL 中添加文件类型以指定 FIELD_OPTIONALLY_ENCLOSED_BY = 属性解决了第一个错误。

但是，当我尝试从 csv 文件导入 NUMBER 值时，我已经使用了 FIELD_OPTIONALLY_ENCLOSED_BY='"'; 但它不起作用。我收到“无法识别数值 '"3922000"'”错误。

我的 .csv 文件示例如下所示：

"3922000","14733370","57256","2","3","2","2","2019-05-23 14:14:44",",00000000",",00000000",",00000000",",00000000","1000,00000000","1000,00000000","1317,50400000","1166,50000000" ,",00000000",",00000000",",00000000",",00000000",",00000000",",00000000",",00000000",",00000000",",00000000",",00000000",",00000000" ,",00000000",",00000000",",00000000",",00000000",",00000000",",00000000","","tcllVeEFPD"

我的 COPY INTO 语句如下：

COPY INTO '..'

   FROM '...'

FILE_FORMAT = (TYPE = CSV 

        STRIP_NULL_VALUES = TRUE 

        FIELD_DELIMITER = ',' 

        SKIP_HEADER = 1  

        error_on_column_count_mismatch=false 

        FIELD_OPTIONALLY_ENCLOSED_BY = '"'

       )

ON_ERROR = "ABORT_STATEMENT";

我感觉 NUMBER 被解释为 STRING。

有没有人解决这个问题？

【参考方案1】：

尝试在 COPY 命令的 FROM 子句中使用子查询，其中列出了每一列并转换相应的列。

例如

COPY INTO '...'
FROM (
    SELECT $1::INTEGER
           $2::FLOAT
           ...
)
FILE_FORMAT = (TYPE = CSV 
        STRIP_NULL_VALUES = TRUE 
        FIELD_DELIMITER = ',' 
        SKIP_HEADER = 1  
        error_on_column_count_mismatch=false 
        FIELD_OPTIONALLY_ENCLOSED_BY = '"'
       )
ON_ERROR = "ABORT_STATEMENT";