NoSuchMethodError: org.apache.flink.api.common.state.OperatorStateStore.getSerializableListState

Posted 2023-01-30 花猫哈哈哈

一、问题描述

将flink版本从1.10.0 升级到 1.12.0，

原pom文件（摘取主要的和flink相关的）

 <properties>
      <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
      <flink.version>1.10.0</flink.version>
      <java.version>1.8</java.version>
      <scala.binary.version>2.11</scala.binary.version>
      <hadoop.version>2.7.4</hadoop.version>
      <maven.compiler.source>$java.version</maven.compiler.source>
      <maven.compiler.target>$java.version</maven.compiler.target>
  </properties>

  <dependencies>
      <!-- Apache Flink dependencies -->
      <!-- These dependencies are provided, because they should not be packaged into the JAR file. -->
      <dependency>
          <groupId>org.apache.flink</groupId>
          <artifactId>flink-java</artifactId>
          <version>$flink.version</version>
          <scope>provided</scope>
      </dependency>
      <dependency>
          <groupId>org.apache.flink</groupId>
          <artifactId>flink-streaming-java_$scala.binary.version</artifactId>
          <version>$flink.version</version>
          <scope>provided</scope>
      </dependency>
      <!-- Add connector dependencies here. They must be in the default scope (compile). -->
      <dependency>
          <groupId>org.apache.flink</groupId>
          <artifactId>flink-connector-kafka-0.11_$scala.binary.version</artifactId>
          <version>$flink.version</version>
      </dependency>
  </dependencies>

<flink.version>1.10.0</flink.version> 替换为 <flink.version>1.12.0</flink.version> 后，

flink-connector-kafka-0.11_2.11 无法加载 1.12.0这个版本，后面我将flink-connector-kafka-0.11_2.11改为其它的版本，服务部署还是报相同的错误，说明版本还是不兼容。

二、解决过程

1、意识到可能是flink-connector-kafka-0.11_2.11 这个版本和其它flink包不兼容，所以尝试替换flink-connector-kafka-0.11_2.11版本为原来的1.10.0，按照maven依赖上推荐的版本升级了其它的包依然没用

 2、找到flink 1.12.0的官方文档，官方文档中1.12.0中移除了kafka 0.11.x connectors

 按照官方文档，移除了对flink-connector-kafka-0.11_2.11的依赖，换了其它的包(flink-sql-connector-kafka_2.11)，写了个更通用的Kafka consumer，再发布成功

新的pom（摘取主要）

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <flink.version>1.12.0</flink.version>
    <java.version>1.8</java.version>
    <scala.binary.version>2.11</scala.binary.version>
    <maven.compiler.source>$java.version</maven.compiler.source>
    <maven.compiler.target>$java.version</maven.compiler.target>

</properties>

<dependencies>
    <!-- Apache Flink dependencies -->
    <!-- These dependencies are provided, because they should not be packaged into the JAR file. -->
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-java</artifactId>
        <version>$flink.version</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-streaming-java_$scala.binary.version</artifactId>
        <version>$flink.version</version>
        <scope>provided</scope>
    </dependency>
    <!-- Add connector dependencies here. They must be in the default scope (compile). -->
    <dependency>
        <groupId>org.apache.flink</groupId>
        <artifactId>flink-sql-connector-kafka_$scala.binary.version</artifactId>
        <version>$flink.version</version>
    </dependency>
</dependencies>