SpringBoot整合spark_sql：报错解决java.lang.NoClassDefFoundError: org/codehaus/janino/InternalCompilerExcept

Posted 2023-03-04 路痴Me

sql连接代码：

public void sqlTest() 

            Dataset<Row> jdbcData = sparkSession.read()
                    .format("jdbc")
                    .option("url", "jdbc:mysql://66.100.172.56:3306/user?useSSL=false&serverTimezone=UTC&allowPublicKeyRetrieval=true&allowMultiQueries=true&useUnicode=true&characterEncoding=UTF-8&useSSL=false&allowPublicKeyRetrieval=true&autoReconnect=true")
                    .option("dbtable", "(SELECT * FROM sys_user) tmp")
                    .option("user", "root")
                    .option("password", "123456")
                    .option("driver", "com.mysql.jdbc.Driver")
                    .load();
            //打印表结构
            jdbcData.printSchema();
            //显示sql查询的结果
            jdbcData.show();

            JavaRDD<Row> rowRDD = jdbcData.javaRDD();
            //输出结果
            System.out.println(rowRDD.collect());

    

所用依赖：

        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.11</artifactId>
            <version>2.4.4</version>
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql -->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.11</artifactId>
            <version>2.4.4</version>
            <scope>provided</scope>
        </dependency>

在使用spark连接数据库后，在调用jdbcData.show()时，出现以下异常:java.lang.ClassNotFoundException: org.codehaus.commons.compiler.UncheckedCompileException

解决方法

加上commons-compiler依赖：

        <dependency>
            <groupId>org.codehaus.janino</groupId>
            <artifactId>commons-compiler</artifactId>
            <version>2.7.8</version>
        </dependency>