Spark Transformations
Posted
tags:
篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了Spark Transformations相关的知识，希望对你有一定的参考价值。
We all know the following fact:
1, RDD are immutable
2, Never modify RDD in place
3, Transform RDD to another RDD
There are 2 different transformations for RDD, one is narrow transformation:
transformations like map, flatMap, filter all are narrow transformation, which means shuffle won‘t happen, so it‘s fast, it‘s speed just depends on:
1, availability of local memory
2, CPU speed
another is wide transfomration:
transformations like groupByKey, reduceByKey, repartition all are wide tranformation, the network speed in shuffle is the key to it‘s speed, so it‘s slower
the final comparison:
以上是关于Spark Transformations的主要内容，如果未能解决你的问题，请参考以下文章