Spark SQL on HIVE

Posted 2020-07-21 王金松

1. SPARK CONF中添加hive-site.xml hive.metastore.uris thrift://master:9083 2. 启动hive元数据 hive --metastore >meta.log 2>&1 & 3. scala>val hiveContext = new org.apache.spark.sql.hive.HiveContext(sc) //hiveContext scala>hiveContext.sql("use hive") datasource(hive) scala>hiveContext.sql("show tables").collect.foreach(println) scala>hiveContext.sql(select count(*) from aaa) scala>sqlContext.read.json(/aaa/bbb/ccc/ddd.json) scala>sqlContext.show()// //create dataframe from dataSource scala>sqlContext.printScheme()