Spring boot 2.3.4 - Kafka 指标在 /actuator/prometheus 中不可见
Posted
技术标签:
【中文标题】Spring boot 2.3.4 - Kafka 指标在 /actuator/prometheus 中不可见【英文标题】:Spring boot 2.3.4 - Kafka metrics are not visible in /actuator/prometheus 【发布时间】:2021-04-23 18:29:43 【问题描述】:我有一个 Spring Boot 应用程序(版本 2.3.4),我正在使用 @KafkaListener 来消费记录。我还使用执行器和千分尺(1.5.5 版)作为指标。
问题是我在 /actuator/prometheus 中看不到 Kafka 指标。 我正在使用以下依赖项:
'org.springframework.boot' version '2.3.4.RELEASE'
implementation group: 'org.springframework.kafka', name: 'spring-kafka', version: '2.5.10.RELEASE'
implementation group: 'org.apache.kafka', name: 'kafka-clients', version: '2.5.1'
并将这些属性添加到 application.yaml:
management:
server:
port: 9091
endpoints:
web:
exposure:
include: "*"
endpoint:
health:
probes:
enabled: true
spring:
jmx:
enabled: true
如果我应该添加任何其他内容以使 kafka 指标在 /actuator/prometheus 中可见,请尝试失败
请注意,当我使用默认 KafkaTemplate 时,这些指标是可见的,但在尝试创建自定义 KafkaTemplate 时,指标会消失:
@SpringBootApplication
public class Application
public static void main(String[] args)
SpringApplication.run(Application.class, args);
@Bean
public ProducerFactory<String, String> customProducerFactory()
Map<String, Object> configProps = new HashMap<>();
configProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9092");
configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, Serdes.String().serializer().getClass().getName());
configProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, Serdes.String().serializer().getClass().getName());
return new DefaultKafkaProducerFactory<>(configProps);
@Bean
public KafkaTemplate<String, String> customProducer()
return new KafkaTemplate<>(customProducerFactory());
@KafkaListener(id = "test", topics = "test_topic")
public void listen(String in)
System.out.println(in);
@Bean
public NewTopic topic()
return TopicBuilder.name("test_topic").partitions(1).replicas(1).build();
@Bean
public ApplicationRunner runner(KafkaTemplate<String, String> template)
return args ->
template.send("test_topic", "foo");
;
【问题讨论】:
有没有人找到过这方面的文档?我能找到的只是这个垃圾,其中包括一些随机的 xml 样本,没有指明将其放在哪里docs.spring.io/spring-integration/reference/html/jmx.html 【参考方案1】:解决方案是在自定义 kafkaTemplate 中添加一个监听器:
@Bean
public ProducerFactory<String, String> customProducerFactory()
Map<String, Object> configProps = new HashMap<>();
configProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "127.0.0.1:9092");
configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, Serdes.String().serializer().getClass().getName());
configProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, Serdes.String().serializer().getClass().getName());
DefaultKafkaProducerFactory<String, String> producerFactory = new DefaultKafkaProducerFactory<>(configProps);
producerFactory.addListener(new MicrometerProducerListener<>(meterRegistry));
@Bean
public KafkaTemplate<String, String> customProducer()
return new KafkaTemplate<>(customProducerFactory());
【讨论】:
【参考方案2】:我刚刚在 Boot 2.4.2 (spring-kafka 2.6.5) 上尝试过,没有任何问题:
@SpringBootApplication
public class So65791799Application
public static void main(String[] args)
SpringApplication.run(So65791799Application.class, args);
@KafkaListener(id = "so65791799", topics = "so65791799")
public void listen(String in)
System.out.println(in);
@Bean
public NewTopic topic()
return TopicBuilder.name("so65791799").partitions(1).replicas(1).build();
@Bean
public ApplicationRunner runner(KafkaTemplate<String, String> template)
return args ->
template.send("so65791799", "foo");
;
server:
port: 9091
management:
endpoints:
web:
exposure:
include: "*"
endpoint:
health:
probes:
enabled: true
spring:
jmx:
enabled: true
kafka:
consumer:
auto-offset-reset: earliest
http://localhost:9091/actuator/prometheus
...
# HELP kafka_consumer_fetch_manager_records_per_request_avg The average number of records in each request
# TYPE kafka_consumer_fetch_manager_records_per_request_avg gauge
kafka_consumer_fetch_manager_records_per_request_avgclient_id="consumer-so65791799-1",kafka_version="2.6.0",spring_id="kafkaConsumerFactory.consumer-so65791799-1", 0.5
...
我把它放回 Boot 2.3.5,它仍然对我有用。
【讨论】:
格雷,感谢您的回复。当使用默认的 KafkaTemplate (与您的示例相同)时,它也对我有用,但是在尝试创建自定义模板时,指标会消失。我编辑了我的问题以包含应用程序代码。以上是关于Spring boot 2.3.4 - Kafka 指标在 /actuator/prometheus 中不可见的主要内容,如果未能解决你的问题,请参考以下文章
docker快速安装kafka,zookeeper ,体验spring-boot-demo-mq-kafka