如何修复 Google DataFlow Pipeline (args) 空指针异常？

Posted 2023-03-25

【中文标题】如何修复 Google DataFlow Pipeline (args) 空指针异常？

【英文标题】：How to fix Google DataFlow Pipeline (args) null pointer exception?

【发布时间】：2016-08-29 20:05:02

【问题描述】：

我正在尝试运行一个非常简单的数据流作业，只是在 BigQuery 中获取一些数据，对其进行一些处理并将其放入一个新的 bigquery 表中

Pipeline p = Pipeline.create( PipelineOptionsFactory.fromArgs(args).withValidation().create()); p.apply(BigQueryIO.Read.fromQuery("SELECT * FROM realtime.status_6_output_11")); p.run();

但是，每当我运行它时，我都会收到以下非常难以描述的 NullPointerException：

Exception in thread "main" java.lang.NullPointerException
    at java.util.regex.Matcher.getTextLength(Matcher.java:1283)
    at java.util.regex.Matcher.reset(Matcher.java:309)
    at java.util.regex.Matcher.<init>(Matcher.java:229)
    at java.util.regex.Pattern.matcher(Pattern.java:1093)
    at com.google.cloud.dataflow.sdk.util.IOChannelUtils.getFactory(IOChannelUtils.java:174)
    at com.google.cloud.dataflow.sdk.io.BigQueryIO$Read$Bound.apply(BigQueryIO.java:553)
    at com.google.cloud.dataflow.sdk.io.BigQueryIO$Read$Bound.apply(BigQueryIO.java:387)
    at com.google.cloud.dataflow.sdk.runners.PipelineRunner.apply(PipelineRunner.java:74)
    at com.google.cloud.dataflow.sdk.runners.DirectPipelineRunner.apply(DirectPipelineRunner.java:247)
    at com.google.cloud.dataflow.sdk.Pipeline.applyInternal(Pipeline.java:367)
    at com.google.cloud.dataflow.sdk.Pipeline.applyTransform(Pipeline.java:274)
    at com.google.cloud.dataflow.sdk.values.PBegin.apply(PBegin.java:47)
    at com.google.cloud.dataflow.sdk.Pipeline.apply(Pipeline.java:156)
    at com.noraway.conductor.NormalizedPipeline.main(NormalizedPipeline.java:42)

I think there's a problem with my command line arguments (don't have any right now) but I'm not sure what that would be.

【问题讨论】：

猜测：args 为空？！

别以为就是这样

你是如何运行它的？你传递论据吗？

【参考方案1】：

BigQuery 似乎缺少要使用的 --tempLocation。模糊的错误消息已作为 https://github.com/GoogleCloudPlatform/DataflowJavaSDK/issues/313 的一部分修复。