Scala RDD 反向键值对

Posted 2023-03-23

【中文标题】Scala RDD 反向键值对

【英文标题】：Scala RDD reverse key value pair

【发布时间】：2021-11-11 10:56:31

【问题描述】：

我有一个结构如下的 RDD 键值对：

scala> val rateMovieByYear = maxRateByYearRDD.join(rateMovie)
rateMovieByYear: org.apache.spark.rdd.RDD[(Double, (String, String))] = MapPartitionsRDD[33] at join at <console>:27

scala> rateMovieByYear.take(5).foreach(println)
(14.0009,(2001,Tortilla Soup))
(5.1258,(1955,Lady and the Tramp))
(12.5,(2013,Chi bi xia: Jue zhan tian xia))
(12.5,(2013,Dante's Inferno: Documentary))
(12.5,(2013,José Rizal))

我想反转一个值作为键和列表值中的键 前任： （玉米饼汤，（14,0009,2001））

感谢您的帮助

【问题讨论】：

您是否尝试过使用map？

非常感谢您的建议，它有效！

【参考方案1】：

我越来越相信 spark RDD 的力量

scala> rateMovieByYear.map case (k, (v1, v2)) => (v2, (k, v1)) 
res0: org.apache.spark.rdd.RDD[(String, (Double, String))] = MapPartitionsRDD[10] at map at <console>:26

scala>

scala> val rateMovieByYearRDD = rateMovieByYear.map case (k, (v1, v2)) => (v2, (k, v1)) 
rateMovieByYearRDD: org.apache.spark.rdd.RDD[(String, (Double, String))] = MapPartitionsRDD[11] at map at <console>:25

scala>

scala> rateMovieByYearRDD.take(5).foreach(println)
(Tortilla Soup,(14.0009,2001))
(Lady and the Tramp,(5.1258,1955))
(Chi bi xia: Jue zhan tian xia,(12.5,2013))
(Dante's Inferno: Documentary,(12.5,2013))
(José Rizal,(12.5,2013))