使用选定列更改表 - debezium 抛出 ArrayIndexOutOfBoundsException

Posted 2023-03-23

【中文标题】使用选定列更改表 - debezium 抛出 ArrayIndexOutOfBoundsException

【英文标题】：Change table with selected columns - debezium throws ArrayIndexOutOfBoundsException

【发布时间】：2020-02-13 19:07:06

【问题描述】：

CDC 表的列数少于源表。当 debezium 尝试在 Kafka 中创建事件时，它失败了：

ArrayIndexOutOfBoundsException。

历史主题具有完整源表架构的快照。 这是 CDC 架构不能与源架构不同的 debezium 限制吗？

This connector will be stopped.\n\tat io.debezium.connector.base.ChangeEventQueue.throwProducerFailureIfPresent(ChangeEventQueue.java:170)\n\tat 
io.debezium.connector.base.ChangeEventQueue.poll(ChangeEventQueue.java:151)\n\tat io.debezium.connector.sqlserver.SqlServerConnectorTask.poll(SqlServerConnectorTask.java:158)\n\tat 
org.apache.kafka.connect.runtime.WorkerSourceTask.poll(WorkerSourceTask.java:245)\n\tat org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:221)\n\tat 
org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:175)\n\tat org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:219)\n\tat 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)\n\tat java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)\n\tat
 java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)\n\tat java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)\n\tat 
 java.base/java.lang.Thread.run(Thread.java:834)\nCaused by: java.lang.ArrayIndexOutOfBoundsException: Index 10 out of bounds for length 10\n\tat io.debezium.relational.TableSchemaBuilder.lambda$createValueGenerator$2(TableSchemaBuilder.java:210)\n\tat
  io.debezium.relational.TableSchema.valueFromColumnData(TableSchema.java:135)\n\tat io.debezium.relational.RelationalChangeRecordEmitter.emitUpdateRecord(RelationalChangeRecordEmitter.java:89)\n\tat 
  io.debezium.relational.RelationalChangeRecordEmitter.emitChangeRecords(RelationalChangeRecordEmitter.java:46)\n\tat 
  io.debezium.pipeline.EventDispatcher.dispatchDataChangeEvent(EventDispatcher.java:125)\n\tat 
  io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.lambda$execute$1(SqlServerStreamingChangeEventSource.java:203)\n\tat 
  io.debezium.jdbc.JdbcConnection.prepareQuery(JdbcConnection.java:485)\n\tat 
  io.debezium.connector.sqlserver.SqlServerConnection.getChangesForTables(SqlServerConnection.java:143)\n\tat 
  io.debezium.connector.sqlserver.SqlServerStreamingChangeEventSource.execute(SqlServerStreamingChangeEventSource.java:137)\n\tat 
  io.debezium.pipeline.ChangeEventSourceCoordinator.lambda$start$0(ChangeEventSourceCoordinator.java:86)\n\t... 5 more\n"],"type":"source"```

【参考方案1】：

捕获表必须捕获源表中定义的所有列。

有一个 WIP PR https://github.com/debezium/debezium/pull/748 消除了这个限制。