Hystrix-dashboard 无法在 Kubernetes 上与 Kubeflix 一起使用

Posted

技术标签:

【中文标题】Hystrix-dashboard 无法在 Kubernetes 上与 Kubeflix 一起使用【英文标题】:Hystrix-dashboard not working with Kubeflix on Kubernetes 【发布时间】:2017-08-28 08:06:57 【问题描述】:

我正在使用 Spring Boot(Fabric8 的 Spring-Clug-Kubernetes)将微服务部署到 Kubernetes,我现在想要一个由 Fabric8 Kubeflix 提供的 Hystrix Dashboard。

我在我的部署上设置了标签:

metadata : 
    labels:
        hystrix.cluster: default
        hystrix.enabled: true
spec:
    template:
        metadata : 
            labels:
                hystrix.cluster: default
                hystrix.enabled: true

它也在我的豆荚上:

metadata : 
    labels:
        hystrix.cluster: default
        hystrix.enabled: true

在我的涡轮服务器 pod 日志中:

2017-03-31T15:23:10.514696068Z 2017-03-31 15:23:10.514  INFO [turbine-server,,,] 1 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Found hosts that have been previously terminated: 0
2017-03-31T15:23:10.514700568Z 2017-03-31 15:23:10.514  INFO [turbine-server,,,] 1 --- [        Timer-0] c.n.t.discovery.InstanceObservable       : Hosts up:0, hosts down: 0

它的 /discovery 端点显示:

Hystrix Endpoints:

涡轮服务器应用程序.yml:

spring:
  application:
    name: turbine-server
turbine:
  instanceUrlSuffix: :80/hystrix.stream
  aggregator:
    clusterConfig: default
InstanceDiscovery:
  impl: io.fabric8.kubeflix.turbine.TurbineDiscovery

在我的微服务中,我只有一个

@EnableCircuitBreaker
@EnableHystrix

在它们的主要应用程序类中。

我将端口 80 设置为涡轮机后缀,因为我有 Kubernetes 服务暴露 pod 的端口 80:

spec: 
    ports: [
        name: default   
        protocol:   TCP,
        port:   80,
        targetPort: 8080,
        nodePort:   32193
    ],
    clusterIP:  10.0.72.62,
    type:   NodePort

当我对我的服务执行 /health 时:

"status": "UP",
"hystrix": 
    "status": "UP"
,

还有 /hystrix.stream :

data: "type":"HystrixCommand","name":"getLabel","group":"LabelController","currentTime":1491222462325,"isCircuitBreakerOpen":false,"errorPercentage":0,"errorCount":0,"requestCount":0,"rollingCountBadRequests":0,"rollingCountCollapsedRequests":0,"rollingCountEmit":0,"rollingCountExceptionsThrown":0,"rollingCountFailure":0,"rollingCountFallbackEmit":0,"rollingCountFallbackFailure":0,"rollingCountFallbackMissing":0,"rollingCountFallbackRejection":0,"rollingCountFallbackSuccess":0,"rollingCountResponsesFromCache":0,"rollingCountSemaphoreRejected":0,"rollingCountShortCircuited":0,"rollingCountSuccess":0,"rollingCountThreadPoolRejected":0,"rollingCountTimeout":0,"currentConcurrentExecutionCount":0,"rollingMaxConcurrentExecutionCount":0,"latencyExecute_mean":0,"latencyExecute":"0":0,"25":0,"50":0,"75":0,"90":0,"95":0,"99":0,"99.5":0,"100":0,"latencyTotal_mean":0,"latencyTotal":"0":0,"25":0,"50":0,"75":0,"90":0,"95":0,"99":0,"99.5":0,"100":0,"propertyValue_circuitBreakerRequestVolumeThreshold":20,"propertyValue_circuitBreakerSleepWindowInMilliseconds":5000,"propertyValue_circuitBreakerErrorThresholdPercentage":50,"propertyValue_circuitBreakerForceOpen":false,"propertyValue_circuitBreakerForceClosed":false,"propertyValue_circuitBreakerEnabled":true,"propertyValue_executionIsolationStrategy":"THREAD","propertyValue_executionIsolationThreadTimeoutInMilliseconds":1000,"propertyValue_executionTimeoutInMilliseconds":1000,"propertyValue_executionIsolationThreadInterruptOnTimeout":true,"propertyValue_executionIsolationThreadPoolKeyOverride":null,"propertyValue_executionIsolationSemaphoreMaxConcurrentRequests":10,"propertyValue_fallbackIsolationSemaphoreMaxConcurrentRequests":10,"propertyValue_metricsRollingStatisticalWindowInMilliseconds":10000,"propertyValue_requestCacheEnabled":true,"propertyValue_requestLogEnabled":true,"reportingHosts":1,"threadPool":"LabelController"
data: "type":"HystrixThreadPool","name":"LabelController","currentTime":1491222462325,"currentActiveCount":0,"currentCompletedTaskCount":2,"currentCorePoolSize":10,"currentLargestPoolSize":2,"currentMaximumPoolSize":10,"currentPoolSize":2,"currentQueueSize":0,"currentTaskCount":2,"rollingCountThreadsExecuted":0,"rollingMaxActiveThreads":0,"rollingCountCommandRejections":0,"propertyValue_queueSizeRejectionThreshold":5,"propertyValue_metricsRollingStatisticalWindowInMilliseconds":10000,"reportingHosts":1

hystrix-dashboard 接收到以下的turber.stream:

"reportingHostsLast10Seconds":0,"name":"meta","type":"meta","timestamp":1491222578286

版本:

Spring Boot : 1.4.1.RELEASE
io.fabric8.kubernetes-client : 1.4.14
io.fabric8.kuberflix.turbine-discovery : 1.0.28
Spring cloud Neflix : 1.2.4.RELEASE
Netflix hystrix : 1.5.6

我的涡轮服务器出了什么问题?为什么它没有从我的微服务中检测到 hystrix.stream ?

【问题讨论】:

【参考方案1】:

好的,我不得不禁用默认的 Eureka 发现:

ribbon.eureka.enabled: false
eureka.client.enabled: false

以便使用 Kubernetes。

instanceUrlSuffix 仍然需要为 :8080/hystrix.stream,因为它需要 Pod 的内部 k8s IP。

【讨论】:

并在 SERVICES 上设置 hystrix 标签!不仅仅是部署和 Pod。 (仅在服务上就足够了,待测试) 我发现在服务上设置hystrix标签不仅够用,而且是必须的。如果您在 pod/deployments 上设置标签,它将不起作用。这很不幸,因为您必须定义一个服务才能让涡轮机收集指标......

以上是关于Hystrix-dashboard 无法在 Kubernetes 上与 Kubeflix 一起使用的主要内容,如果未能解决你的问题,请参考以下文章

使用 FileBasedInstanceDiscovery 的 hystrix-dashboard 涡轮机的问题

微服务和VUE(25): 微服务之Hystrix-dashboard

spring boot 使用Hystrix-dashboard 监控Feign的单个应用

SpringCloud之Hystrix-Dashboard监控,以及踩的坑...

Edgware Feign hystrix-dashboard

SpringCloud系列之二---Feign实现服务间调用,集成Hystrix熔断器Hystrix-Dashboard仪表盘