从 Spring Boot 2.1.6 升级到 2.2.2 时,Prometheus 不导出 Kafka Consumer 数据

Posted

技术标签:

【中文标题】从 Spring Boot 2.1.6 升级到 2.2.2 时,Prometheus 不导出 Kafka Consumer 数据【英文标题】:Prometheus not exporting Kafka Consumer data upon upgrade from Spring Boot 2.1.6 to 2.2.2 【发布时间】:2020-04-17 23:02:26 【问题描述】:

我们将 Spring Boot 版本从 2.1.6 升级到 2.2.2。在 2.1.6 的 Prometheus 端点中看到的某些属性在 2.2.2 中不可见。

kafka_consumer_records_consumed_total_records_total kafka_consumer_records_lag_records kafka_consumer_fetch_latency_max_seconds kafka_consumer_bytes_consumed_total_bytes_total

但是,此数据正在 MBean 中注册。

kafka_consumer_records_consumed_total_records_total 示例

我是否缺少一些配置?如何将 MBean 中可用的这些数据导出到执行器端点(“/prometheus”)?

更新:我们在两个版本中使用的 Prometheus 版本是相同的。

    1.3.1

POM.xml(更改前)

<?xml version="1.0" encoding="UTF-8"?><project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>

<groupId>com.example</groupId>
<artifactId>actuator-demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>

<name>actuator-demo</name>
<description>Spring Boot Actuator Demo Project</description>

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.1.6.RELEASE</version>
    <relativePath/> <!-- lookup parent from repository -->
</parent>

<properties>
    <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
    <java.version>1.8</java.version>
</properties>

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-actuator</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-webflux</artifactId>
    </dependency>

    <!-- Micrometer Prometheus registry  -->
    <dependency>
        <groupId>io.micrometer</groupId>
        <artifactId>micrometer-registry-prometheus</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>

    <dependency>
        <groupId>org.springframework.kafka</groupId>
        <artifactId>spring-kafka</artifactId>
    </dependency>
    <dependency>
        <groupId>org.springframework.kafka</groupId>
        <artifactId>spring-kafka-test</artifactId>
        <scope>test</scope>
        <exclusions>
            <exclusion>
                <artifactId>jackson-databind</artifactId>
                <groupId>com.fasterxml.jackson.core</groupId>
            </exclusion>
        </exclusions>
    </dependency>
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-contract-wiremock</artifactId>
    </dependency>
    <dependency>
        <groupId>io.projectreactor.kafka</groupId>
        <artifactId>reactor-kafka</artifactId>
    </dependency>


</dependencies>

<build>
    <plugins>
        <plugin>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-maven-plugin</artifactId>
        </plugin>
    </plugins>
</build>

POM.xml(改动后——只升级了Spring Boot)

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.example</groupId>
    <artifactId>actuator-demo</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>

    <name>actuator-demo</name>
    <description>Spring Boot Actuator Demo Project</description>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.2.2.RELEASE</version>
        <relativePath/> <!-- lookup parent from repository -->
    </parent>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-actuator</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-webflux</artifactId>
        </dependency>

        <!-- Micrometer Prometheus registry  -->
        <dependency>
            <groupId>io.micrometer</groupId>
            <artifactId>micrometer-registry-prometheus</artifactId>
        </dependency>

        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka-test</artifactId>
            <scope>test</scope>
            <exclusions>
                <exclusion>
                    <artifactId>jackson-databind</artifactId>
                    <groupId>com.fasterxml.jackson.core</groupId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-contract-wiremock</artifactId>
        </dependency>
        <dependency>
            <groupId>io.projectreactor.kafka</groupId>
            <artifactId>reactor-kafka</artifactId>
        </dependency>


    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.springframework.boot</groupId>
                <artifactId>spring-boot-maven-plugin</artifactId>
            </plugin>
        </plugins>
    </build>


</project>

【问题讨论】:

你能报告一下传递版本的变化吗?例如之前和现在的 Micrometer 和 Kafka 客户端的版本是什么? @checketts 升级前后的版本是一样的。 1.3.2.我已经用信息更新了问题。 我为此提出了一个错误。 github.com/spring-projects/spring-boot/issues/19507 【参考方案1】:

Kafka 指标使用 JMX,从 Spring Boot 2.2 开始默认禁用 JMX。您需要启用 JMX 才能使 Kafka Metrics 正常工作,即配置中的 spring.jmx.enabled=true

【讨论】:

以上是关于从 Spring Boot 2.1.6 升级到 2.2.2 时,Prometheus 不导出 Kafka Consumer 数据的主要内容,如果未能解决你的问题,请参考以下文章

Spring Boot - 从 2.2.5 升级到 2.3.0 后验证停止工作

从Spring Boot 1.5升级到2.0

Spring Boot - 从 2.2.5 升级到 2.5.7 后,应用程序无法启动

将 Spring Boot 从 2.4.X 升级到 2.6.X 后无法运行我的 jar

独立资源服务器(Spring Boot 2 + OAuth + JWT)在 Spring-boot 从 1.2.x 升级到 2.x 后给出 UsernameNotFoundException

Spring Boot 2.0干货系列:Spring Boot1.5X升级到2.0指南