spark streaming 异常No output streams registered, so nothing to execute

Posted

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了spark streaming 异常No output streams registered, so nothing to execute相关的知识,希望对你有一定的参考价值。

实现spark streaming demo时,代码:

 1 public static void main (String[] args) {
 2 
 3 
 4 SparkConf conf = new SparkConf().setAppName("Spark_Streaming").setMaster("local");
 5         JavaSparkContext sc = new JavaSparkContext(conf);       
 6         JavaStreamingContext jssc = new JavaStreamingContext(sc, new Duration(2));      
 7         JavaSQLContext sqlCtx = new JavaSQLContext(sc);     
 8 
 9 
10         String[] filters = new String[] {"soc"};
11 
12         JavaReceiverInputDStream<Status> receiverStream = TwitterUtils.createStream(jssc,filters);
13 
14 
15 
16          jssc.start();
17          jssc.awaitTermination();
18 
19 }

运行以上代码会出现异常:No output streams registered, so nothing to execute

原因是当发生输出操作的时候,会触发流计算操作,当在 DStream上没有输出操作或者计算时,通常需要执行某些操作,比如以任意方法:

1 print()
2 foreachRDD(func)
3 saveAsObjectFiles(prefix, [suffix])
4 saveAsTextFiles(prefix, [suffix])
5 saveAsHadoopFiles(prefix, [suffix])

 可参考:http://spark.apache.org/docs/latest/streaming-programming-guide.html#output-operations

以上是关于spark streaming 异常No output streams registered, so nothing to execute的主要内容,如果未能解决你的问题,请参考以下文章

15Spark Streaming源码解读之No Receivers彻底思考

Spark Streaming发行版笔记15:no receivers彻底思考

Spark Streaming源码解读之No Receivers详解

Spark Streaming源码解读之No Receivers详解

第15课:Spark Streaming源码解读之No Receivers彻底思考

[java][spark streamming]java.lang.IllegalArgumentException: requirement failed: No output operations