Spring Cloud Bus 网络中断后无法恢复

Posted 2023-02-16

技术标签:

【中文标题】Spring Cloud Bus 网络中断后无法恢复【英文标题】：Spring Cloud Bus fails to recover after network outage 【发布时间】：2019-05-18 02:16:21 【问题描述】：

我有一个使用 RabbitMQ binder 的 Spring Cloud Stream 应用程序，基于 Spring Boot 1.5 和 Spring Cloud Edgware.SR4。由于某种原因，Spring Cloud Bus 无法从持续至少 30 秒的网络中断（持续时间可能取决于您的环境）。如果网络中断时间足够长，RabbitMQ 已经删除了独占队列，应用程序能够重新声明匿名队列。这似乎是与系统修补/重启 RabbitMQ 服务器相一致的重复事件。

我通过连接到远程 RabbitMQ 服务器并断开笔记本电脑上的网络连接来复制此 sample application 中的错误。 output.txt 包含测试的整个标准输出。

以下是事件概要：

声明 springCloudBus 匿名队列：

2018-12-17 08:37:45.633 DEBUG 13983 --- [           main] o.s.amqp.rabbit.core.RabbitAdmin         : declaring Queue 'springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ'

springCloudBus 匿名队列消费者初始化：

2018-12-17 08:37:45.881 DEBUG 13983 --- [pool-2-thread-7] o.s.a.r.listener.BlockingQueueConsumer   : ConsumeOK: Consumer@503556cb: tags=[amq.ctag-s-IYhxP4bXEjrglFIuQr-w=springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ], channel=Cached Rabbit Channel: AMQChannel(amqp://<my.rabbit.user>@<my.rabbit.ip>:<my.rabbit.port>/notification,1), conn: Proxy@71b0289b Shared Rabbit Connection: SimpleConnection@6f2d3391 [delegate=amqp://<my.rabbit.user>@<my.rabbit.ip>:<my.rabbit.port>/notification, localPort= 52315], acknowledgeMode=AUTO local queue size=0

网络连接中断：

2018-12-17 08:40:23.254 DEBUG 13983 --- [rmx6DqbNo1uPQ-1] o.s.a.r.listener.BlockingQueueConsumer   : Retrieving delivery for Consumer@503556cb: tags=[amq.ctag-s-IYhxP4bXEjrglFIuQr-w=springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ], channel=Cached Rabbit Channel: AMQChannel(amqp://<my.rabbit.user>@<my.rabbit.ip>:<my.rabbit.port>/notification,1), conn: Proxy@71b0289b Shared Rabbit Connection: SimpleConnection@6f2d3391 [delegate=amqp://<my.rabbit.user>@<my.rabbit.ip>:<my.rabbit.port>/notification, localPort= 52315], acknowledgeMode=AUTO local queue size=0
2018-12-17 08:40:23.687 DEBUG 13983 --- [.input.sample-1] o.s.a.r.listener.BlockingQueueConsumer   : Retrieving delivery for Consumer@a03529c: tags=[amq.ctag-3ZNc-38WQH4RfRA6fCmurQ=test.input.sample], channel=Cached Rabbit Channel: AMQChannel(amqp://<my.rabbit.user>@<my.rabbit.ip>:<my.rabbit.port>/notification,2), conn: Proxy@71b0289b Shared Rabbit Connection: SimpleConnection@6f2d3391 [delegate=amqp://<my.rabbit.user>@<my.rabbit.ip>:<my.rabbit.port>/notification, localPort= 52315], acknowledgeMode=AUTO local queue size=0
2018-12-17 08:40:23.724 ERROR 13983 --- [.<my.rabbit.ip>:<my.rabbit.port>] c.r.c.impl.ForgivingExceptionHandler     : An unexpected connection driver error occured

java.net.SocketException: Operation timed out (Read failed)
    at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.8.0_162]
    at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[na:1.8.0_162]
    at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[na:1.8.0_162]
    at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[na:1.8.0_162]
    at sun.security.ssl.InputRecord.readFully(InputRecord.java:465) ~[na:1.8.0_162]
    at sun.security.ssl.InputRecord.read(InputRecord.java:503) ~[na:1.8.0_162]
    at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:983) ~[na:1.8.0_162]
    at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:940) ~[na:1.8.0_162]
    at sun.security.ssl.AppInputStream.read(AppInputStream.java:105) ~[na:1.8.0_162]
    at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) ~[na:1.8.0_162]
    at java.io.BufferedInputStream.read(BufferedInputStream.java:265) ~[na:1.8.0_162]
    at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:288) ~[na:1.8.0_162]
    at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:91) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:164) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:602) ~[amqp-client-4.8.3.jar:4.8.3]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]

网络连接恢复：

2018-12-17 08:41:59.535  INFO 13983 --- [input.sample-12] o.s.a.r.l.SimpleMessageListenerContainer : Restarting Consumer@7ca9d55c: tags=[], channel=null, acknowledgeMode=AUTO local queue size=0
2018-12-17 08:41:59.535 DEBUG 13983 --- [input.sample-12] o.s.a.r.listener.BlockingQueueConsumer   : Closing Rabbit Channel: null
2018-12-17 08:41:59.535 DEBUG 13983 --- [input.sample-13] o.s.a.r.listener.BlockingQueueConsumer   : Starting consumer Consumer@7f088f6f: tags=[], channel=null, acknowledgeMode=AUTO local queue size=0
2018-12-17 08:41:59.864  INFO 13983 --- [mx6DqbNo1uPQ-13] o.s.a.r.c.CachingConnectionFactory       : Created new connection: rabbitConnectionFactory#3db1ce78:23/SimpleConnection@39241539 [delegate=amqp://<my.rabbit.user>@<my.rabbit.ip>:<my.rabbit.port>/notification, localPort= 52463]
2018-12-17 08:41:59.865 DEBUG 13983 --- [mx6DqbNo1uPQ-13] o.s.amqp.rabbit.core.RabbitAdmin         : Initializing declarations
2018-12-17 08:41:59.867 DEBUG 13983 --- [mx6DqbNo1uPQ-13] o.s.amqp.rabbit.core.RabbitAdmin         : Nothing to declare
2018-12-17 08:41:59.867 DEBUG 13983 --- [mx6DqbNo1uPQ-13] o.s.amqp.rabbit.core.RabbitAdmin         : Initializing declarations
2018-12-17 08:41:59.868  INFO 13983 --- [mx6DqbNo1uPQ-13] o.s.amqp.rabbit.core.RabbitAdmin         : Auto-declaring a non-durable, auto-delete, or exclusive Queue (springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ) durable:false, auto-delete:true, exclusive:true. It will be redeclared if the broker stops and is restarted while the connection factory is alive, but all messages will be lost.

尝试从 springCloudBus 匿名队列读取时的消费者异常：

2018-12-17 08:42:17.748 DEBUG 13983 --- [mx6DqbNo1uPQ-13] o.s.a.r.l.SimpleMessageListenerContainer : Consumer raised exception, processing can restart if the connection factory supports it

org.springframework.amqp.AmqpIOException: java.io.IOException
    at org.springframework.amqp.rabbit.support.RabbitExceptionTranslator.convertRabbitAccessException(RabbitExceptionTranslator.java:71) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.connection.RabbitAccessor.convertRabbitAccessException(RabbitAccessor.java:113) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1469) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1419) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.core.RabbitTemplate.execute(RabbitTemplate.java:1395) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.core.RabbitAdmin.initialize(RabbitAdmin.java:579) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.core.RabbitAdmin$11$1.doWithRetry(RabbitAdmin.java:486) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:287) ~[spring-retry-1.2.2.RELEASE.jar:na]
    at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:164) ~[spring-retry-1.2.2.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.core.RabbitAdmin$11.onCreate(RabbitAdmin.java:481) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.connection.CompositeConnectionListener.onCreate(CompositeConnectionListener.java:33) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:594) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils$1.createConnection(ConnectionFactoryUtils.java:90) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.doGetTransactionalResourceHolder(ConnectionFactoryUtils.java:140) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.getTransactionalResourceHolder(ConnectionFactoryUtils.java:76) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:576) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1504) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
Caused by: java.io.IOException: null
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:124) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:120) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:142) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.queueDeclare(ChannelN.java:958) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.queueDeclare(ChannelN.java:53) ~[amqp-client-4.8.3.jar:4.8.3]
    at sun.reflect.GeneratedMethodAccessor157.invoke(Unknown Source) ~[na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_162]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_162]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$CachedChannelInvocationHandler.invoke(CachingConnectionFactory.java:1029) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at com.sun.proxy.$Proxy113.queueDeclare(Unknown Source) ~[na:na]
    at org.springframework.amqp.rabbit.core.RabbitAdmin.declareQueues(RabbitAdmin.java:655) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.core.RabbitAdmin.access$200(RabbitAdmin.java:72) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.core.RabbitAdmin$12.doInRabbit(RabbitAdmin.java:584) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.core.RabbitTemplate.doExecute(RabbitTemplate.java:1463) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    ... 15 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=405, reply-text=RESOURCE_LOCKED - cannot obtain exclusive access to locked queue 'springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ' in vhost 'notification', class-id=50, method-id=10)
    at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:443) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:263) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:136) ~[amqp-client-4.8.3.jar:4.8.3]
    ... 26 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=405, reply-text=RESOURCE_LOCKED - cannot obtain exclusive access to locked queue 'springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ' in vhost 'notification', class-id=50, method-id=10)
    at com.rabbitmq.client.impl.ChannelN.asyncShutdown(ChannelN.java:515) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.processAsync(ChannelN.java:340) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:162) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:109) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection.readFrame(AMQConnection.java:676) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection.access$300(AMQConnection.java:48) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:603) ~[amqp-client-4.8.3.jar:4.8.3]
    ... 1 common frames omitted

最后一个“RESOURCE_LOCKED”异常

2018-12-17 08:42:43.793  WARN 13983 --- [mx6DqbNo1uPQ-15] o.s.a.r.listener.BlockingQueueConsumer   : Queue declaration failed; retries left=2

org.springframework.amqp.rabbit.listener.BlockingQueueConsumer$DeclarationException: Failed to declare queue(s):[springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ]
    at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.attemptPassiveDeclarations(BlockingQueueConsumer.java:706) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:596) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1504) [spring-rabbit-1.7.11.RELEASE.jar:na]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
Caused by: java.io.IOException: null
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:124) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:120) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:142) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:1002) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:53) ~[amqp-client-4.8.3.jar:4.8.3]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_162]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_162]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_162]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_162]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$CachedChannelInvocationHandler.invoke(CachingConnectionFactory.java:1029) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at com.sun.proxy.$Proxy113.queueDeclarePassive(Unknown Source) ~[na:na]
    at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.attemptPassiveDeclarations(BlockingQueueConsumer.java:685) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    ... 3 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=405, reply-text=RESOURCE_LOCKED - cannot obtain exclusive access to locked queue 'springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ' in vhost 'notification', class-id=50, method-id=10)
    at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:443) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:263) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:136) ~[amqp-client-4.8.3.jar:4.8.3]
    ... 12 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=405, reply-text=RESOURCE_LOCKED - cannot obtain exclusive access to locked queue 'springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ' in vhost 'notification', class-id=50, method-id=10)
    at com.rabbitmq.client.impl.ChannelN.asyncShutdown(ChannelN.java:515) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.processAsync(ChannelN.java:340) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:162) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:109) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection.readFrame(AMQConnection.java:676) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection.access$300(AMQConnection.java:48) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:603) ~[amqp-client-4.8.3.jar:4.8.3]
    ... 1 common frames omitted

反复出现的错误：

2018-12-17 08:42:49.781 DEBUG 13983 --- [mx6DqbNo1uPQ-15] o.s.a.r.c.CachingConnectionFactory       : Detected closed channel on exception.  Re-initializing: AMQChannel(amqp://<my.rabbit.user>@<my.rabbit.ip>:<my.rabbit.port>/notification,3)
2018-12-17 08:42:49.800  WARN 13983 --- [mx6DqbNo1uPQ-15] o.s.a.r.listener.BlockingQueueConsumer   : Failed to declare queue: springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ
2018-12-17 08:42:49.801  WARN 13983 --- [mx6DqbNo1uPQ-15] o.s.a.r.listener.BlockingQueueConsumer   : Queue declaration failed; retries left=1

org.springframework.amqp.rabbit.listener.BlockingQueueConsumer$DeclarationException: Failed to declare queue(s):[springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ]
    at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.attemptPassiveDeclarations(BlockingQueueConsumer.java:706) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:596) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1504) [spring-rabbit-1.7.11.RELEASE.jar:na]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
Caused by: java.io.IOException: null
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:124) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:120) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:142) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:1002) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:53) ~[amqp-client-4.8.3.jar:4.8.3]
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_162]
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_162]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_162]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_162]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$CachedChannelInvocationHandler.invoke(CachingConnectionFactory.java:1029) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at com.sun.proxy.$Proxy113.queueDeclarePassive(Unknown Source) ~[na:na]
    at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.attemptPassiveDeclarations(BlockingQueueConsumer.java:685) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    ... 3 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no queue 'springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ' in vhost 'notification', class-id=50, method-id=10)
    at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:443) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:263) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:136) ~[amqp-client-4.8.3.jar:4.8.3]
    ... 12 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no queue 'springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ' in vhost 'notification', class-id=50, method-id=10)
    at com.rabbitmq.client.impl.ChannelN.asyncShutdown(ChannelN.java:515) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.processAsync(ChannelN.java:340) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:162) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:109) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection.readFrame(AMQConnection.java:676) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection.access$300(AMQConnection.java:48) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:603) ~[amqp-client-4.8.3.jar:4.8.3]
    ... 1 common frames omitted


2018-12-17 08:44:04.527 ERROR 13983 --- [mx6DqbNo1uPQ-18] o.s.a.r.l.SimpleMessageListenerContainer : Consumer received fatal=false exception on startup

org.springframework.amqp.rabbit.listener.QueuesNotAvailableException: Cannot prepare queue for listener. Either the queue doesn't exist or the broker will not allow us to use it.
    at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:628) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:1504) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
Caused by: org.springframework.amqp.rabbit.listener.BlockingQueueConsumer$DeclarationException: Failed to declare queue(s):[springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ]
    at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.attemptPassiveDeclarations(BlockingQueueConsumer.java:706) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:596) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    ... 2 common frames omitted
Caused by: java.io.IOException: null
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:124) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:120) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:142) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:1002) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.queueDeclarePassive(ChannelN.java:53) ~[amqp-client-4.8.3.jar:4.8.3]
    at sun.reflect.GeneratedMethodAccessor161.invoke(Unknown Source) ~[na:na]
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_162]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_162]
    at org.springframework.amqp.rabbit.connection.CachingConnectionFactory$CachedChannelInvocationHandler.invoke(CachingConnectionFactory.java:1029) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    at com.sun.proxy.$Proxy113.queueDeclarePassive(Unknown Source) ~[na:na]
    at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.attemptPassiveDeclarations(BlockingQueueConsumer.java:685) ~[spring-rabbit-1.7.11.RELEASE.jar:na]
    ... 3 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no queue 'springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ' in vhost 'notification', class-id=50, method-id=10)
    at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:66) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:36) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:443) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:263) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:136) ~[amqp-client-4.8.3.jar:4.8.3]
    ... 11 common frames omitted
Caused by: com.rabbitmq.client.ShutdownSignalException: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no queue 'springCloudBus.anonymous.p6jqufA8Trmx6DqbNo1uPQ' in vhost 'notification', class-id=50, method-id=10)
    at com.rabbitmq.client.impl.ChannelN.asyncShutdown(ChannelN.java:515) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.ChannelN.processAsync(ChannelN.java:340) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.handleCompleteInboundCommand(AMQChannel.java:162) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQChannel.handleFrame(AMQChannel.java:109) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection.readFrame(AMQConnection.java:676) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection.access$300(AMQConnection.java:48) ~[amqp-client-4.8.3.jar:4.8.3]
    at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:603) ~[amqp-client-4.8.3.jar:4.8.3]
    ... 1 common frames omitted

【问题讨论】：

【参考方案1】：

这意味着经纪人看到原始消费者死亡的时间过长；由于队列是独占的，这可以防止恢复在重新建立连接时重新声明队列。

RabbitAdmin 具有以下重试配置（当没有明确定义时）：

            this.retryTemplate = new RetryTemplate();
            this.retryTemplate.setRetryPolicy(new SimpleRetryPolicy(5));
            ExponentialBackOffPolicy backOffPolicy = new ExponentialBackOffPolicy();
            backOffPolicy.setInitialInterval(1000);
            backOffPolicy.setMultiplier(2.0);
            backOffPolicy.setMaxInterval(5000);
            this.retryTemplate.setBackOffPolicy(backOffPolicy);

所以它会尝试 5 次并在 1 + 2 + 4 + 5 (12) 秒后放弃。

Spring Cloud Stream 目前不支持配置其管理员重试配置。我建议您针对rabbit binder 提出问题。

【讨论】：

加里，谢谢你的解释。我在 GitHub 上打开了一个问题。 (github.com/spring-cloud/spring-cloud-stream-binder-rabbit/…)

以上是关于Spring Cloud Bus 网络中断后无法恢复的主要内容，如果未能解决你的问题，请参考以下文章

Spring Cloud Bus with rabbit - 应用程序无法启动

Spring Cloud学习记录 08Spring Cloud Bus服务总线

干货｜Spring Cloud Bus 消息总线介绍

通过总线机制实现自动刷新客户端配置(Consul,Spring Cloud Config,Spring Cloud Bus)

问题 spring-cloud-config 和 spring-cloud-bus

spring cloud-stream 和 spring cloud-bus 有啥区别？