将字符串列类型转换为小数

Posted 2021-05-07

我有一个数据框，其中的数据为StringType。像这样

+----------------+
|            colx|
+----------------+
|    537118968.00|
|     25530934.00|
|      1613378.00|
|   1585857645.00|
|     19566217.00|
|     97978779.00|
|        46817.00|
|               0|
|               0|
|       721030.00|
+----------------+

我正在尝试将列转换为十进制类型，但它仅返回空值

dF.select(col("colx").cast("decimal(30,0)")).show

+----+
|colx|
+----+
|null|
|null|
|null|
|null|
|null|
|null|
|null|
|null|
|null|
|null|
+----+

我无法理解这里的错误。 Spark版本2.4.2

答案

无问题v2.4.2

scala> import org.apache.spark.sql.SparkSession

implicit lazy val spark: SparkSession = {
  val spark = SparkSession.builder
    .master("local[*]")
    .appName("SparkTestApp")
    .config("spark.driver.host", "localhost")
    .getOrCreate()
  spark.sparkContext.setLogLevel("ERROR")
  spark
}

import spark.implicits._
import org.apache.spark.sql.functions.col

val df = Seq(
  "537118968.00",
  "25530934.00",
  "1613378.00",
  "1585857645.00",
  "19566217.00",
  "97978779.00",
  "46817.00",
  "0",
  "0",
  "721030.00"
).toDF()

df.select(col("value").cast("decimal(30,0)")).show
import org.apache.spark.sql.SparkSession

scala>      |      |      |      |      |      |      |      | spark: org.apache.spark.sql.SparkSession = <lazy>

scala> import spark.implicits._

scala> import org.apache.spark.sql.functions.col

scala>      |      |      |      |      |      |      |      |      |      |      | df: org.apache.spark.sql.DataFrame = [value: string]

scala> +----------+
|     value|
+----------+
| 537118968|
|  25530934|
|   1613378|
|1585857645|
|  19566217|
|  97978779|
|     46817|
|         0|
|         0|
|    721030|
+----------+