无法将 Spring Boot 指标发布到 GCP 堆栈驱动程序

Posted

技术标签:

【中文标题】无法将 Spring Boot 指标发布到 GCP 堆栈驱动程序【英文标题】:Unable to publish spring boot metrics to GCP stackdriver 【发布时间】:2020-08-13 10:54:20 【问题描述】:

我正在尝试将在 GCP 计算引擎中运行的 spring boot(2.2.6.RELEASE) 应用程序的指标发布到 stackdriver。我添加了micrometer-registry-stackdriver:1.3.6spring-boot-actuator的依赖。 Stackdriver 指标客户端配置按照micrometer link 中的说明完成。 在虚拟机中我安装了云监控代理as mentioned here。

在运行应用程序时,我收到错误 io.grpc.StatusRuntimeException: INTERNAL: An internal error occurred.

如果我缺少任何其他配置,谁能告诉我?

2020-04-29 10:19:16.248  INFO 14905 --- [           main] o.s.web.context.ContextLoader            : Root WebApplicationContext: initialization completed in 3608
ms
2020-04-29 10:19:18.085  INFO 14905 --- [           main] i.m.s.StackdriverMeterRegistry           : publishing metrics to stackdriver every 1m
2020-04-29 10:19:20.058  INFO 14905 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Initializing ExecutorService 'applicationTaskExecutor'
2020-04-29 10:19:20.735  INFO 14905 --- [           main] o.s.b.a.e.web.EndpointLinksResolver      : Exposing 13 endpoint(s) beneath base path '/actuator'
2020-04-29 10:19:20.897  INFO 14905 --- [           main] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2020-04-29 10:19:20.908  INFO 14905 --- [           main] com.example.demo.DemoApplication         : Started DemoApplication in 9.375 seconds (JVM running for 10.
417)
2020-04-29 10:19:27.548  INFO 14905 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/]       : Initializing Spring DispatcherServlet 'dispatcherServlet'
2020-04-29 10:19:27.549  INFO 14905 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Initializing Servlet 'dispatcherServlet'
2020-04-29 10:19:27.568  INFO 14905 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet        : Completed initialization in 19 ms
2020-04-29 10:20:39.832  WARN 14905 --- [trics-publisher] i.m.s.StackdriverMeterRegistry           : failed to send metrics to Stackdriver

com.google.api.gax.rpc.InternalException: io.grpc.StatusRuntimeException: INTERNAL: An internal error occurred.
        at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:67) ~[gax-1.48.1.jar!/:1.48.1]
        at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:72) ~[gax-grpc-1.48.1.jar!/:1.48.1]
        at com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:60) ~[gax-grpc-1.48.1.jar!/:1.48.1]
        at com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97) ~[gax-grpc-1.48.1.jar!/:1.48.1]
        at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:68) ~[api-common-1.8.1.jar!/:na]
        at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:982) ~[guava-28.0-android.jar!/:na]
        at com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:30) ~[guava-28.0-android.jar!/:na]
        at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1138) ~[guava-28.0-android.jar!/:na]
        at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:957) ~[guava-28.0-android.jar!/:na]
        at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:748) ~[guava-28.0-android.jar!/:na]
        at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:515) ~[grpc-stub-1.23.0.jar!/:1.23.0]
        at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:490) ~[grpc-stub-1.23.0.jar!/:1.23.0]
        at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) ~[grpc-api-1.23.0.jar!/:1.23.0]
        at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) ~[grpc-api-1.23.0.jar!/:1.23.0]
        at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) ~[grpc-api-1.23.0.jar!/:1.23.0]
        at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:700) ~[grpc-core-1.23.0.jar!/:1.23.0]
        at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39) ~[grpc-api-1.23.0.jar!/:1.23.0]
        at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23) ~[grpc-api-1.23.0.jar!/:1.23.0]
        at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40) ~[grpc-api-1.23.0.jar!/:1.23.0]
        at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:399) ~[grpc-core-1.23.0.jar!/:1.23.0]
        at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:510) ~[grpc-core-1.23.0.jar!/:1.23.0]
        at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:66) ~[grpc-core-1.23.0.jar!/:1.23.0]
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:630) ~[grpc-core-1.23.0.jar!/:1.23.0]
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$700(ClientCallImpl.java:518) ~[grpc-core-1.23.0.jar!/:1.23.0]
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:692) ~[grpc-core-1.23.0.jar!/:1.23.0]
        at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:681) ~[grpc-core-1.23.0.jar!/:1.23.0]
        at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37) ~[grpc-core-1.23.0.jar!/:1.23.0]
        at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123) ~[grpc-core-1.23.0.jar!/:1.23.0]
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_252]
                at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_252]
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_252]
                at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_252]
                at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[na:1.8.0_252]
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[na:1.8.0_252]
                at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_252]
                Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
                        at com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57) ~[gax-1.48.1.jar!/:1.48.1]
                        at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112) ~[gax-1.48.1.jar!/:1.48.1]
                        at com.google.cloud.monitoring.v3.MetricServiceClient.createTimeSeries(MetricServiceClient.java:1156) ~[google-cloud-monitoring-1.93.0.jar!/:1.93.
        0]
                        at io.micrometer.stackdriver.StackdriverMeterRegistry.publish(StackdriverMeterRegistry.java:176) ~[micrometer-registry-stackdriver-1.3.6.jar!/:1.3
        .6]
                        at io.micrometer.core.instrument.push.PushMeterRegistry.publishSafely(PushMeterRegistry.java:48) [micrometer-core-1.3.6.jar!/:1.3.6]
                        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_252]
                        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) ~[na:1.8.0_252]
                        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_252]
                        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) ~[na:1.8.0_252]
                        ... 3 common frames omitted
        Caused by: io.grpc.StatusRuntimeException: INTERNAL: An internal error occurred.
                at io.grpc.Status.asRuntimeException(Status.java:533) ~[grpc-api-1.23.0.jar!/:1.23.0]
                ... 24 common frames omitted
        2020-04-29 10:20:55.181  INFO 14905 --- [nio-8080-exec-5] o.apache.coyote.http11.Http11Processor   : Error parsing HTTP request header
         Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level.

更新:-在将千分尺版本升级到 1.5.1

后,我能够将指标发布到 stackdriver

【问题讨论】:

【参考方案1】:

首先,检查你是否有io.micrometer:micrometer-core 依赖。其次,根据我的经验,让你的meterRegistry() bean 返回MeterRegistry 而不是StackdriverMeterRegistry

@Bean
public MeterRegistry meterRegistry() 
    final StackdriverConfig stackdriverConfig = new StackdriverConfig() 
        @Override
        public String projectId() 
            return "your-project-name";
        

        @Override
        public String get(final String key) 
            return null; //this field is ignored
        
    ;
    return StackdriverMeterRegistry.builder(stackdriverConfig).build();

另外,我希望您正确配置了management.* 属性。这是我的例子

management:
  endpoints:
    metrics:
      enabled: true
    prometheus:
      enabled: true
    web:
      exposure:
        include: '*'
  metrics:
    tags:
      application: your-app
    export:
      prometheus:
        enabled: true
    stackdriver:
      enabled: true
      step: 1m

【讨论】:

感谢您的回复。我可以通过将千分尺版本升级到1.5.1 来解决这个问题。 StackdriverMeterRegistryMeterRegistry 的子类。所以改变meterRegistry()的返回类型没有任何区别,但我同意编程到父类或接口是好的。

以上是关于无法将 Spring Boot 指标发布到 GCP 堆栈驱动程序的主要内容,如果未能解决你的问题,请参考以下文章

Spring Boot GCP:将 PubSub 应用程序部署到 App Engine 标准环境时出现“Google 凭据”异常

在 GCP PubSub 和 Spring Boot 中捕获发布错误

将带有 JMS (ActiveMQ) 的 Spring Boot 应用程序迁移到 GCP 的 pub-sub

将 GCP 订阅者与 Spring Boot 集成会导致内存致命错误

使用 Spring Boot 2.0 时将指标导出到文件

无法使用自定义内存指标自动缩放 GCP 实例