kettle连接sparksql

Posted 2022-08-05 weixin_46108337

kettle连接sparksql报错信息为没有合适的驱动是因为版本问题或者是jar包不齐等问题，本人试过用kettle8.2以及kettle9.1连接都会报此类错，需要的jar包以及版本，放在lib目录里。此版本对应的是kettle9.0

spark-hive-thriftserver_2.11-2.4.3.jar
spark-avro_2.11-4.0.0.jar
spark-catalyst_2.11-2.4.3.jar
spark-core_2.11-2.4.3.jar
spark-hive_2.11-2.4.3.jar
spark-kubernetes_2.11-2.4.3.jar
spark-launcher_2.11-2.4.3.jar
spark-mllib-local_2.11-2.4.3.jar
spark-sql_2.11-2.4.3.jar
spark-network-common_2.11-2.4.3.jar
spark-sql_2.11-2.4.3.jar
hive-cli-1.2.1.spark2.jar
hive-exec-1.2.1.spark2.jar
hive-jdbc-1.2.1.spark2.jar
hive-metastore-1.2.1.spark2.jar
protobuf-java-2.5.0.jar
libthrift-0.9.3.jar
hadoop-common-2.7.3.jar
hadoop-hdfs-2.7.3.jar
hadoop-hdfs-client-2.7.3.jar
hadoop-mapreduce-client-core-2.7.3.jar
hadoop-yarn-client-2.7.3.jar

除此之位需要注意的是用kettle数据库连接是选择hive2/3 端口我这边设置的是10015 因端口10000被hive占用 故sparksql设置的端口是10015

测试连接成功