spring cloud gateway 并发问题：java.lang.NullPointerException: null

Posted 2022-10-30 回归心灵

tags:

篇首语：本文由小常识网(cha138.com)小编为大家整理，主要介绍了spring cloud gateway 并发问题：java.lang.NullPointerException: null相关的知识，希望对你有一定的参考价值。

问题描述

基于 spring cloud gateway 做的网关服务在运行一段时间后会报 NullPointerException 异常。异常日志如下：

2022-03-11 15:06:37.933 ERROR 1 --- [DiscoveryClient-CacheRefreshExecutor-0] com.netflix.discovery.DiscoveryClient    : Cannot fetch registry from server

reactor.core.Exceptions$ErrorCallbackNotImplemented: java.lang.NullPointerException
Caused by: java.lang.NullPointerException: null
	at org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.addWeightConfig(WeightCalculatorWebFilter.java:204)
	Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
	|_ checkpoint ⇢ com.xdcloud.gateway.acpect.ManageLoginInterceptor [DefaultWebFilterChain]
	|_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
	|_ checkpoint ⇢ com.alibaba.csp.sentinel.adapter.spring.webflux.SentinelWebFluxFilter [DefaultWebFilterChain]
	|_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
	|_ checkpoint ⇢ HTTP POST "/powerBank/get" [ExceptionHandlingWebHandler]
	|_ checkpoint ⇢ com.xdcloud.gateway.acpect.ManageLoginInterceptor [DefaultWebFilterChain]
	|_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
	|_ checkpoint ⇢ com.alibaba.csp.sentinel.adapter.spring.webflux.SentinelWebFluxFilter [DefaultWebFilterChain]
	|_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
	|_ checkpoint ⇢ HTTP POST "/powerBank/get" [ExceptionHandlingWebHandler]
	.....
	|_ checkpoint ⇢ com.xdcloud.gateway.acpect.ManageLoginInterceptor [DefaultWebFilterChain]
	|_ checkpoint ⇢ com.xdcloud.gateway.acpect.ManageLoginInterceptor [DefaultWebFilterChain]
	|_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
	|_ checkpoint ⇢ com.xdcloud.gateway.acpect.ManageLoginInterceptor [DefaultWebFilterChain]
	|_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
Stack trace:
		at org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.addWeightConfig(WeightCalculatorWebFilter.java:204)
		at org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.handle(WeightCalculatorWebFilter.java:162)
		at org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.onApplicationEvent(WeightCalculatorWebFilter.java:138)
		at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
		at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
		at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
		at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:403)
		at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:360)
		at org.springframework.cloud.gateway.support.ConfigurationService$AbstractBuilder.bind(ConfigurationService.java:278)
		at org.springframework.cloud.gateway.route.RouteDefinitionRouteLocator.lookup(RouteDefinitionRouteLocator.java:270)
		at org.springframework.cloud.gateway.route.RouteDefinitionRouteLocator.combinePredicates(RouteDefinitionRouteLocator.java:242)
		at org.springframework.cloud.gateway.route.RouteDefinitionRouteLocator.convertToRoute(RouteDefinitionRouteLocator.java:162)
		at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.java:100)
		at reactor.core.publisher.FluxFlatMap$FlatMapMain.drainLoop(FluxFlatMap.java:693)
		at reactor.core.publisher.FluxFlatMap$FlatMapMain.drain(FluxFlatMap.java:569)
		at reactor.core.publisher.FluxFlatMap$FlatMapInner.onSubscribe(FluxFlatMap.java:953)
		at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:161)
		at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:86)
		at reactor.core.publisher.Flux.subscribe(Flux.java:8186)
		at reactor.core.publisher.FluxFlatMap$FlatMapMain.onNext(FluxFlatMap.java:418)
		at reactor.core.publisher.FluxIterable$IterableSubscription.slowPath(FluxIterable.java:267)
		at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:225)
		at reactor.core.publisher.FluxFlatMap$FlatMapMain.onSubscribe(FluxFlatMap.java:363)
		at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:161)
		at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:86)
		at reactor.core.publisher.Flux.subscribe(Flux.java:8186)
		at reactor.core.publisher.FluxFlatMap$FlatMapMain.onNext(FluxFlatMap.java:418)
		at reactor.core.publisher.FluxIterable$IterableSubscription.slowPath(FluxIterable.java:267)
		at reactor.core.publisher.FluxIterable$IterableSubscription.request(FluxIterable.java:225)
		at reactor.core.publisher.FluxFlatMap$FlatMapMain.onSubscribe(FluxFlatMap.java:363)
		at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:161)
		at reactor.core.publisher.FluxIterable.subscribe(FluxIterable.java:86)
		at reactor.core.publisher.InternalFluxOperator.subscribe(InternalFluxOperator.java:53)
		at reactor.core.publisher.FluxDefer.subscribe(FluxDefer.java:54)
		at reactor.core.publisher.Flux.subscribe(Flux.java:8186)
		at reactor.core.publisher.Flux.subscribeWith(Flux.java:8350)
		at reactor.core.publisher.Flux.subscribe(Flux.java:8157)
		at reactor.core.publisher.Flux.subscribe(Flux.java:8084)
		at reactor.core.publisher.Flux.subscribe(Flux.java:8002)
		at org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.lambda$onApplicationEvent$0(WeightCalculatorWebFilter.java:145)
		at org.springframework.beans.factory.ObjectProvider.ifAvailable(ObjectProvider.java:93)
		at org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.onApplicationEvent(WeightCalculatorWebFilter.java:145)
		at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
		at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
		at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
		at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:403)
		at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:360)
		at org.springframework.cloud.gateway.route.RouteRefreshListener.reset(RouteRefreshListener.java:68)
		at org.springframework.cloud.gateway.route.RouteRefreshListener.resetIfNeeded(RouteRefreshListener.java:63)
		at org.springframework.cloud.gateway.route.RouteRefreshListener.onApplicationEvent(RouteRefreshListener.java:57)
		at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
		at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
		at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
		at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:403)
		at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:360)
		at org.springframework.cloud.netflix.eureka.CloudEurekaClient.onCacheRefreshed(CloudEurekaClient.java:123)
		at com.netflix.discovery.DiscoveryClient.fetchRegistry(DiscoveryClient.java:999)
		at com.netflix.discovery.DiscoveryClient.refreshRegistry(DiscoveryClient.java:1497)
		at com.netflix.discovery.DiscoveryClient$CacheRefreshThread.run(DiscoveryClient.java:1464)
		at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
		at java.util.concurrent.FutureTask.run(FutureTask.java:266)
		at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
		at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
		at java.lang.Thread.run(Thread.java:748)

问题分析

根据异常栈信息 org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.addWeightConfig(WeightCalculatorWebFilter.java:204) 可以找到框架源代码中执行的具体方法如下：

注意 spring cloud gateway 版本：

<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-gateway-core</artifactId>
	<version>2.2.0.RELEASE</version>
</dependency>

204 行代码空指针异常，也就是 Double range = previousRange + currentWeight; 语句中 previousRange 或者 currentWeight 值为 null。经过代码分析，在单线程的情况下，两个变量的值都不可能为 null。但是当多个线程进入此方法时，previousRange 变量就可能存在值为 null 的情况。请注意 config 对象是缓存在 ConcurrentHashMap 中的，用于表示每个 group 中不同路由的权重配置。当一个线程执行到 204 行时，此时另一个线程执行到 196 行，清空 config.rages list 数组元素。先看下 config.ranges.clear(); 方法的执行

public void clear() 
        modCount++;

        // clear to let GC do its work
        for (int i = 0; i < size; i++)
            elementData[i] = null;

        size = 0;

ArrayList 的 clear 方法会 for 循环给每一个元素赋值为 null，然后在修改 list 中元素的数量为 0 。

如果当一个线程执行 for 循环时，另一个线程执行 204 行代码获取 list 中的元素，就可能得到一个 null。也就会报空指针异常。

问题复现

由上述理论分析 addWeightConfig 方法存在存在并发问题，但是问题一直很难复现，程序也只有运行很长一段时间才报错。经过较长时间的问题跟踪分析，发现当一个路由组（group）配置很多个路由（route）信息时，就会增加 clear 清除的时长，同时加上不断的刷下路由信息，重新计算路由权重，也就会增大并发问题的可能性。因此为一个路由组添加了几百个路由信息，程序运行几分钟后就出现了空指针异常。

同时也出现了 java.lang.IndexOutOfBoundsException 异常，异常信息如下：

2022-03-11 18:00:21.168 ERROR 1 --- [or-http-epoll-2] a.w.r.e.AbstractErrorWebExceptionHandler : [5269561b-2172]  500 Server Error for HTTP POST "/powerBank/get?authToken=oseus8x"

java.lang.IndexOutOfBoundsException: Index: 87, Size: 240
        at java.util.ArrayList.rangeCheck(ArrayList.java:659)
        Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException: 
Error has been observed at the following site(s):
        |_ checkpoint ⇢ com.xdcloud.gateway.acpect.ManageLoginInterceptor [DefaultWebFilterChain]
        |_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.alibaba.csp.sentinel.adapter.spring.webflux.SentinelWebFluxFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ HTTP POST "/apollo/1.0/inner/powerBank/get?authToken=odysseus8x4dt2gt" [ExceptionHandlingWebHandler]
        |_ checkpoint ⇢ com.xdcloud.gateway.acpect.ManageLoginInterceptor [DefaultWebFilterChain]
        |_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.alibaba.csp.sentinel.adapter.spring.webflux.SentinelWebFluxFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ HTTP POST "/apollo/1.0/inner/powerBank/get?authToken=odysseus8x4dt2gt" [ExceptionHandlingWebHandler]
        |_ checkpoint ⇢ com.xdcloud.gateway.acpect.ManageLoginInterceptor [DefaultWebFilterChain]
        |_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.alibaba.csp.sentinel.adapter.spring.webflux.SentinelWebFluxFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ HTTP POST "/apollo/1.0/inner/powerBank/get?authToken=odysseus8x4dt2gt" [ExceptionHandlingWebHandler]
        |_ checkpoint ⇢ com.xdcloud.gateway.acpect.ManageLoginInterceptor [DefaultWebFilterChain]
        |_ checkpoint ⇢ org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ com.alibaba.csp.sentinel.adapter.spring.webflux.SentinelWebFluxFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ org.springframework.boot.actuate.metrics.web.reactive.server.MetricsWebFilter [DefaultWebFilterChain]
        |_ checkpoint ⇢ HTTP POST "/apollo/1.0/inner/powerBank/get?authToken=odysseus8x4dt2gt" [ExceptionHandlingWebHandler]
Stack trace:
                at java.util.ArrayList.rangeCheck(ArrayList.java:659)
                at java.util.ArrayList.get(ArrayList.java:435)
                at org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.addWeightConfig(WeightCalculatorWebFilter.java:203)
                at org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.handle(WeightCalculatorWebFilter.java:162)
                at org.springframework.cloud.gateway.filter.WeightCalculatorWebFilter.onApplicationEvent(WeightCalculatorWebFilter.java:138)
                at org.springframework.context.event.SimpleApplicationEventMulticaster.doInvokeListener(SimpleApplicationEventMulticaster.java:172)
                at org.springframework.context.event.SimpleApplicationEventMulticaster.invokeListener(SimpleApplicationEventMulticaster.java:165)
                at org.springframework.context.event.SimpleApplicationEventMulticaster.multicastEvent(SimpleApplicationEventMulticaster.java:139)
                at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:403)
                at org.springframework.context.support.AbstractApplicationContext.publishEvent(AbstractApplicationContext.java:360)
                at org.springframework.cloud.gateway.support.ConfigurationService$AbstractBuilder.bind(ConfigurationService.java:278)
                at org.springframework.cloud.gateway.route.RouteDefinitionRouteLocator.lookup(RouteDefinitionRouteLocator.java:270)
                at org.springframework.cloud.gateway.route.RouteDefinitionRouteLocator.combinePredicates(RouteDefinitionRouteLocator.java:242)
                at org.springframework.cloud.gateway.route.RouteDefinitionRouteLocator.convertToRoute(RouteDefinitionRouteLocator.java:162)
                at reactor.core.publisher.FluxMap$MapSubscriber.onNext(FluxMap.javaspring cloud gateway 并发问题：java.lang.NullPointerException: null
 Spring Cloud Gateway 限流操作
 最全面的改造Zuul网关为Spring Cloud Gateway（包含Zuul核心实现和Spring Cloud Gateway核心实现）
 Spring Cloud Gateway 之限流操作
 Spring Cloud Gateway 之限流操作
 spring cloud 2.x版本 Gateway熔断限流教程