Kafka 连接：提供了配置 XXX，但不是 AdminClientConfig 中的已知配置

Posted 2023-03-10

【中文标题】Kafka 连接：提供了配置 XXX，但不是 AdminClientConfig 中的已知配置

【英文标题】：Kafka connect: The configuration XXX was supplied but isn't a known config in AdminClientConfig

【发布时间】：2019-08-22 15:42:04

【问题描述】：

在启动 Kafka-Connect 时，我看到了很多警告

10:33:56.706 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'config.storage.topic' was supplied but isn't a known config.
10:33:56.707 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'group.id' was supplied but isn't a known config.
10:33:56.708 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'status.storage.topic' was supplied but isn't a known config.
10:33:56.709 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'internal.key.converter.schemas.enable' was supplied but isn't a known config.
10:33:56.710 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'config.storage.replication.factor' was supplied but isn't a known config.
10:33:56.710 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'offset.flush.interval.ms' was supplied but isn't a known config.
10:33:56.711 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'key.converter.schemas.enable' was supplied but isn't a known config.
10:33:56.712 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'internal.key.converter' was supplied but isn't a known config.
10:33:56.712 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'internal.value.converter.schemas.enable' was supplied but isn't a known config.
10:33:56.713 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'status.storage.replication.factor' was supplied but isn't a known config.
10:33:56.713 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'value.converter.schemas.enable' was supplied but isn't a known config.
10:33:56.714 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'internal.value.converter' was supplied but isn't a known config.
10:33:56.714 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'offset.storage.replication.factor' was supplied but isn't a known config.
10:33:56.715 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'offset.storage.topic' was supplied but isn't a known config.
10:33:56.715 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'value.converter' was supplied but isn't a known config.
10:33:56.716 [DistributedHerder] WARN  org.apache.kafka.clients.admin.AdminClientConfig - The configuration 'key.converter' was supplied but isn't a known config.

有些帖子说这是因为主题创建被禁用，但在我的情况下不是这样。 存储主题仍然被创建。

另一个提到它的版本不匹配，这也不是我的情况。 kafka broker版本为1.0.0，Kafka-Connect同下面的pom.xml。这些配置是在 Kafka-Connect 启动期间提供的

<properties>
      <!-->
      Confluent version must match Kafka version
      https://docs.confluent.io/current/installation/versions-interoperability.html
      <-->
        <kafka.version>1.0.0</kafka.version>
        <confluent.version>5.1.2</confluent.version>
        <debezium-connector-mongodb.version>0.7.4</debezium-connector-mongodb.version>
        <logging.version>2.11.2</logging.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>connect-json</artifactId>
            <version>$kafka.version</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>connect-runtime</artifactId>
            <version>$kafka.version</version>
            <scope>runtime</scope>
            <exclusions>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>io.debezium</groupId>
            <artifactId>debezium-connector-mongodb</artifactId>
            <version>$debezium-connector-mongodb.version</version>
        </dependency>
        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>connect-file</artifactId>
            <version>$kafka.version</version>
            <scope>runtime</scope>
        </dependency>
        <dependency>
            <groupId>io.confluent</groupId>
            <artifactId>kafka-connect-jdbc</artifactId>
            <version>$confluent.version</version>
            <scope>runtime</scope>
            <exclusions>
                <exclusion>
                    <groupId>log4j</groupId>
                    <artifactId>log4j</artifactId>
                </exclusion>
                <exclusion>
                    <groupId>org.slf4j</groupId>
                    <artifactId>slf4j-log4j12</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

还有什么我应该检查以修复这些警告？我可以看到数据是通过 Kafka-Connect 传递的，但仍然担心这些警告

【参考方案1】：

这些只是WARNings，可以忽略。在未来的版本中，它们将被正确抑制：https://github.com/apache/kafka/pull/5876

【讨论】：

我认为是因为那些常量不在Kafka Client(AdminClientConfig)库中，但是在Kafka-Connect中调用它们是什么原因？

在整个 Kafka 领域，跟踪用户提供的配置（以某种方式）而不是完全抑制这些警告可能很有用；如果用户故意提供配置并期望它会改变语义或处理，它可以很好地暗示您的期望可能不会成功：p ...可能是最初的意图

我仍然在 kafka 2.6.0 中看到这些。忽略确实不是一个好的答案 - 日志应该是干净的，这样监控团队才能对有效的警告做出反应。

@dlipofsky 我引用的 PR 仍然开放，所以我想如果你想继续，可以跟进 :)