Spark IMF传奇行动第17课Transformations实战总结

Posted 2020-06-07

今晚听了王家林老师的Spark IMF传奇行动第17课Transformations实战，作业是用SCALA写cogroup：

def main(args: Array[String]): Unit = {
    val sc = sparkContext("Transformations")

    cogroupTrans(sc)

    sc.stop() 
}
def cogroupTrans(sc:SparkContext): Unit ={
    val stuNames = Array(
      Tuple2(1,"Spark"),
      Tuple2(2,"Tecc"),
      Tuple2(3,"Hadoop")
    )
    val stuScores = Array(
      Tuple2(1,100),
      Tuple2(1,99),
      Tuple2(2,95),
      Tuple2(3,65)
    )
    val names = sc.parallelize(stuNames)
    val scores = sc.parallelize(stuScores)
    val stuNameAndScore = names.cogroup(scores)
    stuNameAndScore.collect().foreach(println)
}

后续课程可以参照新浪微博 王家林_DT大数据梦工厂：http://weibo.com/ilovepains

王家林  中国Spark第一人,微信公共号DT_Spark

 

转发请写明出处。