Postgres 使用 debezium 创建复制槽失败

Posted 2023-03-23

【中文标题】Postgres 使用 debezium 创建复制槽失败

【英文标题】：Postgres using debezium Creation of replication slot failed

【发布时间】：2021-06-29 07:34:53

【问题描述】：

我使用 postgres 配置 debezium 如下代码，当我启动我的 Spring Boot 应用程序时，出现错误 Creation of replication slot failed

@Bean
public io.debezium.config.Configuration connector() 
    return io.debezium.config.Configuration.create()
            .with(EmbeddedEngine.CONNECTOR_CLASS, PostgresConnector.class)
            .with(EmbeddedEngine.OFFSET_STORAGE,  FileOffsetBackingStore.class)
            .with(EmbeddedEngine.OFFSET_STORAGE_FILE_FILENAME, "offset.dat")
            .with(EmbeddedEngine.OFFSET_FLUSH_INTERVAL_MS, 60000)
            .with("name", "test-postgres-connector")
            .with(RelationalDatabaseConnectorConfig.SERVER_NAME, "172.26.113.123:5455-test")
            .with(RelationalDatabaseConnectorConfig.HOSTNAME, "172.26.113.123")
            .with(RelationalDatabaseConnectorConfig.PORT, "5455")
            .with(RelationalDatabaseConnectorConfig.USER, "test")
            .with(RelationalDatabaseConnectorConfig.PASSWORD, "test")
            .with(RelationalDatabaseConnectorConfig.DATABASE_NAME, "test")
            .with(RelationalDatabaseConnectorConfig.TABLE_INCLUDE_LIST, "question").build();



2021-04-02 12:58:56.817 ERROR 14672 --- [pool-5-thread-1] io.debezium.embedded.EmbeddedEngine      : Unable to initialize and start connector's task class 'io.debezium.connector.postgresql.PostgresConnectorTask' with config: name=student-postgres-connector, connector.class=io.debezium.connector.postgresql.PostgresConnector, database.port=5455, offset.storage=org.apache.kafka.connect.storage.FileOffsetBackingStore, table.include.list=question, database.user=pfe, database.hostname=172.26.113.123, offset.storage.file.filename=offset.dat, database.password=********, offset.flush.interval.ms=60000, database.server.name=172.26.113.123:5455-pfe, database.dbname=pfe

io.debezium.DebeziumException: Creation of replication slot failed
    at io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:134) ~[debezium-connector-postgres-1.4.2.Final.jar:1.4.2.Final]
    at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:106) ~[debezium-core-1.4.2.Final.jar:1.4.2.Final]
    at io.debezium.embedded.EmbeddedEngine.run(EmbeddedEngine.java:758) ~[debezium-embedded-1.4.2.Final.jar:1.4.2.Final]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_232]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_232]
    at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_232]
Caused by: org.postgresql.util.PSQLException: ERROR: logical decoding requires wal_level >= logical
    at org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2440) ~[postgresql-42.2.5.jar:42.2.5]
    at org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2183) ~[postgresql-42.2.5.jar:42.2.5]
    at org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:308) ~[postgresql-42.2.5.jar:42.2.5]
    at org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:441) ~[postgresql-42.2.5.jar:42.2.5]
    at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:365) ~[postgresql-42.2.5.jar:42.2.5]
    at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:307) ~[postgresql-42.2.5.jar:42.2.5]
    at org.postgresql.jdbc.PgStatement.executeCachedSql(PgStatement.java:293) ~[postgresql-42.2.5.jar:42.2.5]
    at org.postgresql.jdbc.PgStatement.executeWithFlags(PgStatement.java:270) ~[postgresql-42.2.5.jar:42.2.5]
    at org.postgresql.jdbc.PgStatement.execute(PgStatement.java:266) ~[postgresql-42.2.5.jar:42.2.5]
    at io.debezium.connector.postgresql.connection.PostgresReplicationConnection.createReplicationSlot(PostgresReplicationConnection.java:352) ~[debezium-connector-postgres-1.4.2.Final.jar:1.4.2.Final]
    at io.debezium.connector.postgresql.PostgresConnectorTask.start(PostgresConnectorTask.java:127) ~[debezium-connector-postgres-1.4.2.Final.jar:1.4.2.Final]
    ... 5 common frames omitted

【参考方案1】：

看起来您的 postgres 实例使用了 wal_level 而不是 logical。

Debezium 需要 wal_level 逻辑：

https://www.postgresql.org/docs/9.6/runtime-config-wal.html

在课堂上看一下 postgres 连接器的内部：

io.debezium.connector.postgresql.PostgresConnector.java

【讨论】：

澄清一下：wal_level 不是通过 Debezium 设置的，而是必须在 postgres.conf 中为数据库服务器配置（重启）。见debezium.io/documentation/reference/connectors/…