spark streaming 异常No output streams registered, so nothing to execute

Posted 2020-06-13

实现spark streaming demo时，代码：

 1 public static void main (String[] args) {
 2 
 3 
 4 SparkConf conf = new SparkConf().setAppName("Spark_Streaming").setMaster("local");
 5         JavaSparkContext sc = new JavaSparkContext(conf);       
 6         JavaStreamingContext jssc = new JavaStreamingContext(sc, new Duration(2));      
 7         JavaSQLContext sqlCtx = new JavaSQLContext(sc);     
 8 
 9 
10         String[] filters = new String[] {"soc"};
11 
12         JavaReceiverInputDStream<Status> receiverStream = TwitterUtils.createStream(jssc,filters);
13 
14 
15 
16          jssc.start();
17          jssc.awaitTermination();
18 
19 }

运行以上代码会出现异常：No output streams registered, so nothing to execute

原因是当发生输出操作的时候，会触发流计算操作，当在 DStream上没有输出操作或者计算时，通常需要执行某些操作，比如以任意方法：

1 print()
2 foreachRDD(func)
3 saveAsObjectFiles(prefix, [suffix])
4 saveAsTextFiles(prefix, [suffix])
5 saveAsHadoopFiles(prefix, [suffix])

 可参考：http://spark.apache.org/docs/latest/streaming-programming-guide.html#output-operations