无法为 SqlServer 连接器设置 debezium 嵌入式引擎

Posted 2023-03-23

【中文标题】无法为 SqlServer 连接器设置 debezium 嵌入式引擎

【英文标题】：Unable to setup an debezium-embedded engine for SqlServer Connector

【发布时间】：2020-09-28 21:34:28

【问题描述】：

尝试使用以下配置为 SqlServer 连接器设置 debezium 嵌入式引擎

config = Configuration.create()
 .with("connector.class", "io.debezium.connector.sqlserver.SqlServerConnector")
 .with("offset.storage", "org.apache.kafka.connect.storage.FileOffsetBackingStore")
 .with("offset.storage.file.filename", "/path/to/storage/offset.dat")
 .with("offset.flush.interval.ms", "60000")
 .with("name", "localSqlServerconnector")
 .with("database.hostname", "localhost")
 .with("database.port", "1433")
 .with("database.user", "sa")
 .with("database.password", "pwd@123")
 .with("database.server.id", "12345")
 .with("database.server.name", "localSqlServer")
 .with("database.history", "io.debezium.relational.history.FileDatabaseHistory")
 .with("database.history.file.filename", "/path/to/storage/dbhistory.dat")
 .with("database.history.kafka.topic", "localmysqlServer.dbo")
 .with("database.dbname", "sampledb")
 .with("table.whitelist", "dbo.student")
 .build();

engine = EmbeddedEngine.create().using(config).notifying(this::handleEvent).build();

ExecutorService executor = Executors.newSingleThreadExecutor();
executor.execute(engine);

以下是用于为 SqlServer 连接器建立 debezium 嵌入式引擎设置的依赖项。

<dependency>
  <groupId>io.debezium</groupId>
  <artifactId>debezium-embedded</artifactId>
  <version>0.8.3.Final</version>
</dependency>

<dependency>
  <groupId>io.debezium</groupId>
  <artifactId>debezium-connector-sqlserver</artifactId>
  <version>0.9.0.Final</version>
</dependency>

以下是我尝试启动 debezium 嵌入式引擎时发生的错误

java.lang.NoClassDefFoundError: io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig
    at java.lang.ClassLoader.defineClassImpl(Native Method) ~[na:1.8.0_202]
    at java.lang.ClassLoader.defineClass(ClassLoader.java:379) ~[na:1.8.0_202]
    at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142) ~[na:1.8.0_202]
    at java.net.URLClassLoader.defineClass(URLClassLoader.java:682) ~[na:1.8.0_202]
    at java.net.URLClassLoader.access$400(URLClassLoader.java:89) ~[na:1.8.0_202]
    at java.net.URLClassLoader$ClassFinder.run(URLClassLoader.java:1086) ~[na:1.8.0_202]
    at java.security.AccessController.doPrivileged(AccessController.java:739) ~[na:1.8.0_202]
    at java.net.URLClassLoader.findClass(URLClassLoader.java:589) ~[na:1.8.0_202]
    at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:934) ~[na:1.8.0_202]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:879) ~[na:1.8.0_202]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[na:1.8.0_202]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:862) ~[na:1.8.0_202]
    at io.debezium.connector.sqlserver.SqlServerConnectorTask.getAllConfigurationFields(SqlServerConnectorTask.java:220) ~[debezium-connector-sqlserver-0.9.0.Final.jar:0.9.0.Final]
    at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:36) ~[debezium-core-0.8.3.Final.jar:0.8.3.Final]
    at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:677) ~[debezium-embedded-0.8.3.Final.jar:0.8.3.Final]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_202]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_202]
    at java.lang.Thread.run(Thread.java:813) [na:1.8.0_202]
Caused by: java.lang.ClassNotFoundException: io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig
    at java.net.URLClassLoader.findClass(URLClassLoader.java:591) ~[na:1.8.0_202]
    at java.lang.ClassLoader.loadClassHelper(ClassLoader.java:934) ~[na:1.8.0_202]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:879) ~[na:1.8.0_202]
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) ~[na:1.8.0_202]
    at java.lang.ClassLoader.loadClass(ClassLoader.java:862) ~[na:1.8.0_202]
    ... 18 common frames omitted

如果有任何遗漏，请告诉我。谢谢。

【参考方案1】：

为了避免 API 之间的冲突，保持相同组 id 的版本相同。 你错过了debezium-coredependecy

<!-- https://mvnrepository.com/artifact/io.debezium/debezium-core -->
<dependency>
    <groupId>io.debezium</groupId>
    <artifactId>debezium-core</artifactId>
    <version>1.1.2.Final</version>
</dependency>

<dependency>
  <groupId>io.debezium</groupId>
  <artifactId>debezium-embedded</artifactId>
  <version>1.1.2.Final</version>
</dependency>

<dependency>
  <groupId>io.debezium</groupId>
  <artifactId>debezium-connector-sqlserver</artifactId>
  <version>1.1.2.Final</version>
</dependency>

还将具有相同组 id 的所有工件的版本更改为 1.1.2.Final 还将相同的 jar 添加到运行时环境的类路径中

【讨论】：

感谢@QuickSilver。即使添加了debezium-core 依赖项，也会遇到同样的问题。

这更多的是运行时错误，所以还要将 jar 添加到您的 debezium-embedded engine 的类路径 @KolaGangadhar

但使用类似的io.debezium.config.Configuration，它适用于 MySQL 连接器。

问题中附加的配置是否一切正常？正如我在控制台中看到的以下错误。错误 365784 --- [pool-1-thread-1] io.debezium.embedded.EmbeddedEngine：无法使用配置初始化和启动连接器的任务类 'io.debezium.connector.sqlserver.SqlServerConnectorTask'：name=localSqlServerconnector,. .....

将debezium-embedded engine 添加到类路径后仍然是同样的问题。