Kafka Streams：topic.compression.type不是已知的配置

Posted 2021-04-26

在Kafka Streams中添加压缩配置，与此link类似：

properties.put(StreamsConfig.topicPrefix(TopicConfig.COMPRESSION_TYPE_CONFIG), "snappy");

但我在日志中看到以下WARN消息：

[2018-11-19 23:52:10,158] WARN The configuration 'topic.compression.type' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
[2018-11-19 23:52:10,158] WARN The configuration 'topic.compression.type' was supplied but isn't a known config. (org.apache.kafka.clients.admin.AdminClientConfig)
[2018-11-19 23:52:10,194] WARN The configuration 'topic.compression.type' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)
[2018-11-19 23:52:10,194] WARN The configuration 'topic.compression.type' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)
[2018-11-19 23:52:10,209] WARN The configuration 'topic.compression.type' was supplied but isn't a known config. (org.apache.kafka.clients.producer.ProducerConfig)
[2018-11-19 23:52:10,209] WARN The configuration 'topic.compression.type' was supplied but isn't a known config. (org.apache.kafka.clients.producer.ProducerConfig)
[2018-11-19 23:52:10,221] WARN The configuration 'topic.compression.type' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)
[2018-11-19 23:52:10,221] WARN The configuration 'topic.compression.type' was supplied but isn't a known config. (org.apache.kafka.clients.consumer.ConsumerConfig)

这个设置不值得尊敬吗？它表达得不好吗？

答案

您可以忽略这些警告。 “问题”是，主题配置参数被复制到内部使用的客户端的配置中，并且它们不知道参数名称，因此不知道它的WARN。但是，必须在客户端上设置这些配置，因为客户端会将这些配置“转发”到代码库中知道如何使用它们的其他部分。

这有点不幸和一个已知的问题（但不容易解决）。这个问题有JIRA：https://issues.apache.org/jira/browse/KAFKA-6793