debezium postgres 找不到`io/debezium/util/IoUtil`
Posted
技术标签:
【中文标题】debezium postgres 找不到`io/debezium/util/IoUtil`【英文标题】:debezium postgres cant find `io/debezium/util/IoUtil` 【发布时间】:2020-12-03 10:21:23 【问题描述】:我在使用 docker 时尝试使用 postgres debezium 连接器运行 kafka connect 时遇到问题,它找不到位于 debezium-core
jar 中的某个类。
我已下载 debezium postgres 插件并将其内容复制到我定义的 plugin.path
路径 (/kafka/plugins
)。
即使 kafka connect 加载了第一个 debezium 核心和 api,错误仍然出现。
这是日志:
kafka-connect_1 | [2020-08-13 14:06:13,543] INFO Scanning for plugin classes. This might take a moment ... (org.apache.kafka.connect.cli.ConnectDistributed)
kafka-connect_1 | [2020-08-13 14:06:13,564] INFO Loading plugin from: /kafka/plugins/debezium-api-1.3.0.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,628] INFO Registered loader: PluginClassLoaderpluginLocation=file:/kafka/plugins/debezium-api-1.3.0.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,628] INFO Added plugin 'org.apache.kafka.connect.connector.policy.AllConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,628] INFO Added plugin 'org.apache.kafka.connect.connector.policy.PrincipalConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,629] INFO Added plugin 'org.apache.kafka.connect.connector.policy.NoneConnectorClientConfigOverridePolicy' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,629] INFO Loading plugin from: /kafka/plugins/debezium-core-1.3.0.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,762] INFO Registered loader: PluginClassLoaderpluginLocation=file:/kafka/plugins/debezium-core-1.3.0.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,762] INFO Added plugin 'io.debezium.converters.ByteBufferConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,762] INFO Added plugin 'io.debezium.converters.CloudEventsConverter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,763] INFO Added plugin 'io.debezium.transforms.outbox.EventRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,763] INFO Added plugin 'io.debezium.transforms.ExtractNewRecordState' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,763] INFO Added plugin 'io.debezium.transforms.ContentBasedRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,763] INFO Added plugin 'io.debezium.transforms.ByLogicalTableRouter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,763] INFO Added plugin 'io.debezium.transforms.Filter' (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,763] INFO Loading plugin from: /kafka/plugins/protobuf-java-3.8.0.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,879] INFO Registered loader: PluginClassLoaderpluginLocation=file:/kafka/plugins/protobuf-java-3.8.0.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,879] INFO Loading plugin from: /kafka/plugins/debezium-connector-postgres-1.3.0.jar (org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader)
kafka-connect_1 | [2020-08-13 14:06:13,917] ERROR Stopping due to error (org.apache.kafka.connect.cli.ConnectDistributed)
kafka-connect_1 | java.lang.NoClassDefFoundError: io/debezium/util/IoUtil
kafka-connect_1 | at io.debezium.connector.postgresql.Module.<clinit>(Module.java:19)
kafka-connect_1 | at io.debezium.connector.postgresql.PostgresConnector.version(PostgresConnector.java:47)
kafka-connect_1 | at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:380)
kafka-connect_1 | at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.versionFor(DelegatingClassLoader.java:385)
kafka-connect_1 | at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.getPluginDesc(DelegatingClassLoader.java:355)
kafka-connect_1 | at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanPluginPath(DelegatingClassLoader.java:328)
kafka-connect_1 | at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.scanUrlsAndAddPlugins(DelegatingClassLoader.java:261)
kafka-connect_1 | at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.registerPlugin(DelegatingClassLoader.java:253)
kafka-connect_1 | at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initPluginLoader(DelegatingClassLoader.java:222)
kafka-connect_1 | at org.apache.kafka.connect.runtime.isolation.DelegatingClassLoader.initLoaders(DelegatingClassLoader.java:199)
kafka-connect_1 | at org.apache.kafka.connect.runtime.isolation.Plugins.<init>(Plugins.java:60)
kafka-connect_1 | at org.apache.kafka.connect.cli.ConnectDistributed.startConnect(ConnectDistributed.java:91)
kafka-connect_1 | at org.apache.kafka.connect.cli.ConnectDistributed.main(ConnectDistributed.java:78)
kafka-connect_1 | Caused by: java.lang.ClassNotFoundException: io.debezium.util.IoUtil
kafka-connect_1 | at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
kafka-connect_1 | at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
kafka-connect_1 | at org.apache.kafka.connect.runtime.isolation.PluginClassLoader.loadClass(PluginClassLoader.java:104)
kafka-connect_1 | at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
我已经尝试过在 *** 和其他网站上找到的其他问题中提出的不同解决方案,但似乎没有任何帮助,原因是某些插件已被识别并成功加载,但 @987654325 @一不是...
这个问题有什么解决办法吗?谢谢!
【问题讨论】:
【参考方案1】:解决方案是将整个文件夹原样复制到容器中,而不是将每个文件都复制到插件路径中。
在我的 docker 文件中:
而不是执行COPY ./docker/kafka-connect/debezium-connector-postgres /kafka/plugins
,
我已将包含插件的debezium-connector-postgres
文件夹放入另一个名为connectors
的文件夹中,该文件夹位于./docker/
中,
然后该操作对COPY ./docker/kafka-connect/connectors /kafka/plugins
执行更改,这会在容器的/kafka/plugins
内生成一个新文件夹。
【讨论】:
以上是关于debezium postgres 找不到`io/debezium/util/IoUtil`的主要内容,如果未能解决你的问题,请参考以下文章
table.include.list 配置参数在 Debezium Postgres 连接器中不起作用
使用debeziumkafka-connect将postgres数据实时同步到kafka中
debezium 是不是支持捕获 postgres 模式更改事件?