How to convert matrix to RDD[Vector] in spark

Posted msw0529

tags:

篇首语:本文由小常识网(cha138.com)小编为大家整理,主要介绍了How to convert matrix to RDD[Vector] in spark相关的知识,希望对你有一定的参考价值。

The matrix is generated from SVD, and I am using the results from SVD to do clustering analysis.

 if your clustering only supports RDD as its input, here‘s how you can do the transformation

  def toRDD(sc :SparkContext,m: Matrix): RDD[Vector] = {
        val columns: Iterator[Array[Double]] = m.toArray.grouped(m.numRows)
//        val rows: Seq[Array[Double]] = columns.toSeq // Skip this if you want a column-major RDD.
        val rows: Seq[Seq[Double]] = columns.toSeq.transpose // Skip this if you want a column-major RDD.
        val vectors: Seq[DenseVector] = rows.map(row => new DenseVector(row.toArray))
        sc.parallelize(vectors)

  

以上是关于How to convert matrix to RDD[Vector] in spark的主要内容,如果未能解决你的问题,请参考以下文章