spark1.统计句子中特定内容
Posted 小毛驴
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了spark1.统计句子中特定内容相关的知识，希望对你有一定的参考价值。
val logFile = "./README.md" // Should be some file on your server.
val conf = new SparkConf().setAppName("Simple Application").setMaster("local")
val sc = new SparkContext(conf)
val logData = sc.textFile(logFile, 2).cache()
// val numAs = logData.filter(line => line.contains("h")).count()
// val numBs = logData.filter(line => line.contains("j")).count()
var params = List("h","j","c","w");
var searchAnylisay = params.map(item => logData.filter(line => line.contains(item)).count() )
println("searchAnylisay length : %s,".format(searchAnylisay.length))
searchAnylisay.foreach( x => println(x))
以上是关于spark1.统计句子中特定内容的主要内容，如果未能解决你的问题，请参考以下文章