spark streaming 异常No output streams registered, so nothing to execute
Posted
tags:
篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了spark streaming 异常No output streams registered, so nothing to execute相关的知识,希望对你有一定的参考价值。
实现spark streaming demo时,代码:
1 public static void main (String[] args) { 2 3 4 SparkConf conf = new SparkConf().setAppName("Spark_Streaming").setMaster("local"); 5 JavaSparkContext sc = new JavaSparkContext(conf); 6 JavaStreamingContext jssc = new JavaStreamingContext(sc, new Duration(2)); 7 JavaSQLContext sqlCtx = new JavaSQLContext(sc); 8 9 10 String[] filters = new String[] {"soc"}; 11 12 JavaReceiverInputDStream<Status> receiverStream = TwitterUtils.createStream(jssc,filters); 13 14 15 16 jssc.start(); 17 jssc.awaitTermination(); 18 19 }
运行以上代码会出现异常:No output streams registered, so nothing to execute
原因是当发生输出操作的时候,会触发流计算操作,当在 DStream上没有输出操作或者计算时,通常需要执行某些操作,比如以任意方法:
1 print() 2 foreachRDD(func) 3 saveAsObjectFiles(prefix, [suffix]) 4 saveAsTextFiles(prefix, [suffix]) 5 saveAsHadoopFiles(prefix, [suffix])
可参考:http://spark.apache.org/docs/latest/streaming-programming-guide.html#output-operations
以上是关于spark streaming 异常No output streams registered, so nothing to execute的主要内容,如果未能解决你的问题,请参考以下文章
15Spark Streaming源码解读之No Receivers彻底思考
Spark Streaming发行版笔记15:no receivers彻底思考
Spark Streaming源码解读之No Receivers详解
Spark Streaming源码解读之No Receivers详解
第15课:Spark Streaming源码解读之No Receivers彻底思考
[java][spark streamming]java.lang.IllegalArgumentException: requirement failed: No output operations