RDD中cache和persist的区别

Posted 2022-04-03 timssd

 

RDD中cache和persist的区别

  

 

通过观察RDD.scala源代码即可知道cache和persist的区别：

def persist(newLevel: StorageLevel): this.type = 　　if (storageLevel != StorageLevel.NONE && newLevel != storageLevel) 　　　　throw new UnsupportedOperationException( "Cannot change storage level of an RDD after it was already assigned a level") 　　 　　sc.persistRDD(this) 　　sc.cleaner.foreach(_.registerRDDForCleanup(this)) 　　storageLevel = newLevel 　　this /** Persist this RDD with the default storage level (`MEMORY_ONLY`). */ def persist(): this.type = persist(StorageLevel.MEMORY_ONLY)   /** Persist this RDD with the default storage level (`MEMORY_ONLY`). */ def cache(): this.type = persist()