BigQuery 作业状态已完成，但未使用气流插入任何行

Posted 2023-03-25

【中文标题】BigQuery 作业状态已完成，但未使用气流插入任何行

【英文标题】：BigQuery job status DONE but no row is inserted using airflow

【发布时间】：2019-09-17 13:02:23

【问题描述】：

我已经创建了一个使用 Apache 气流将数据插入 Big Query 的数据管道，但是在运行了几次之后，没有任何行被插入到 Bigquery 中并且 QueryJob 状态仍然是 DONE

[2019-09-17 18:22:33,398] logging_mixin.py:95 INFO - Job d6ceaa36-5917-4b49-a89c-d292c30cf707 is currently in state PENDING
[2019-09-17 18:22:34,360] logging_mixin.py:95 INFO - Job 582f745e-44f4-43d9-854c-8bbed20c4e5a is currently in state DONE
[2019-09-17 18:22:35,548] logging_mixin.py:95 INFO - Job b3ad09ed-37fb-4cfa-ba41-2922f5dfb35d is currently in state DONE
[2019-09-17 18:22:36,369] logging_mixin.py:95 INFO - Job 8501bb2b-6ee4-40a0-831f-63772cf55cb2 is currently in state DONE
[2019-09-17 18:22:37,159] logging_mixin.py:95 INFO - Job 565634c7-fa9f-474d-8227-a7ebf839a954 is currently in state DONE
[2019-09-17 18:22:38,011] logging_mixin.py:95 INFO - Job 4fa2ea5e-5d4d-4caa-ad9d-aa1f6326db86 is currently in state DONE
[2019-09-17 18:22:38,931] logging_mixin.py:95 INFO - Job cf27c305-5be3-40ea-a224-4bc288291b5d is currently in state DONE
[2019-09-17 18:22:39,852] logging_mixin.py:95 INFO - Job f384c582-7b81-4bc9-bb53-aef854dabf5a is currently in state DONE
[2019-09-17 18:22:40,672] logging_mixin.py:95 INFO - Job 391699c0-ed03-4d08-9325-8342525ef645 is currently in state DONE
[2019-09-17 18:22:41,697] logging_mixin.py:95 INFO - Job 2051192a-07bb-4075-ab4b-cb90b7e1012a is currently in state DONE
[2019-09-17 18:22:42,515] logging_mixin.py:95 INFO - Job f99abee5-884a-412f-be23-2d0fea56dabd is currently in state DONE
[2019-09-17 18:22:43,287] logging_mixin.py:95 INFO - Job 7a39757b-6086-49fc-b8ae-93cccce9add4 is currently in state DONE
[2019-09-17 18:22:44,157] logging_mixin.py:95 INFO - Job 344fbbb7-7922-4177-ac20-024062e58b03 is currently in state DONE
[2019-09-17 18:22:45,080] logging_mixin.py:95 INFO - Job acb1cb2d-bec5-4d0a-a237-936e41f9cc90 is currently in state DONE
[2019-09-17 18:22:45,925] logging_mixin.py:95 INFO - Job f194d6e0-c224-45bc-a3ad-29f0dfcc2d60 is currently in state DONE

【参考方案1】：

DONE 并不意味着它已完成插入，它意味着作业过程已完成，您可能会遇到错误或成功。

【讨论】：

知道如何确保插入是否完成