基于 Spring WebFlux/Reactive Mongo 的应用程序打开到 mongo db 的多个连接

Posted 2023-02-26

【中文标题】基于 Spring WebFlux/Reactive Mongo 的应用程序打开到 mongo db 的多个连接

【英文标题】：Spring WebFlux/Reactive Mongo based application opens multiple connections to mongo db

【发布时间】：2020-04-03 16:41:45

【问题描述】：

我有一个简单的 web 应用程序，带有 spring boot 2.x.x 、用于响应式 API 的 Spring WebFlux 和响应式 Mongo 存储库。 在应用程序中，我有一个与 get API 关联的@Tailablequery，以无限观察保存在 DB 中的更改。而且我有一个发布 API 来将数据推送到数据库中。 但是应用程序会随机打开多个到 mongo 的连接。有时这些连接会被重用。

日志

Opened connection [connectionIdlocalValue:4, serverValue:342] to localhost:27017
Opened connection [connectionIdlocalValue:5, serverValue:344] to localhost:27017

等等.. 所以我添加了一个配置类来限制每个主机的连接

  @Configuration
public class MongoConfig 

    @Value("$spring.data.mongodb.maxConnectionIdleTime")
    private int  maxConnectionIdleTime;
    @Value("$spring.data.mongodb.connectionsPerHost")
    private int connectionsPerHost;
    @Value("$spring.data.mongodb.minConnectionsPerHost")
    private int minConnectionsPerHost;
    @Value("$spring.data.mongodb.socketTimeout")
    private int socketTimeout;


    @Bean
    public MongoClientOptions mongoOptions() 
        return MongoClientOptions.builder()
                .maxConnectionIdleTime(this.maxConnectionIdleTime)
                .connectionsPerHost(this.connectionsPerHost)
                .minConnectionsPerHost(this.minConnectionsPerHost)
                .socketTimeout(this.socketTimeout)
                .build();

    

现在，当我运行 Get api（无限期观察数据库）时，应用程序仍会打开多个到 DB 的连接，但由于空闲超时而关闭了其中的几个。

日志：

Opened connection [connectionIdlocalValue:6, serverValue:345] to localhost:27017
Closed connection [connectionIdlocalValue:6, serverValue:345] to localhost:27017 because it is past its maximum allowed idle time.
Opened connection [connectionIdlocalValue:7, serverValue:348] to localhost:27017
Closed connection [connectionIdlocalValue:7, serverValue:348] to localhost:27017 because it is past its maximum allowed idle time.
Opened connection [connectionIdlocalValue:8, serverValue:351] to localhost:27017
Closed connection [connectionIdlocalValue:8, serverValue:351] to localhost:27017 because it is past its maximum allowed idle time.
Opened connection [connectionIdlocalValue:9, serverValue:354] to localhost:27017

编辑

进一步观察，它正在打开一个连接，然后在 60 秒后关闭它，如果该连接没有被使用。例如，在下面的日志中，本地 id 为 5 和 6 的连接被用于通过 api 跟踪 mongo 事件，剩余的连接只是一分钟后关闭的空闲连接。 EDIT 将 mongo 配置更改为 ->

 server:
    port: 8081
spring:
    data:
        mongodb:
            database: LocalMongo
            uri: mongodb://localhost:27017/LocalMongo?maxpoolsize=5
            maxConnectionIdleTime: 15000
            connectionsPerHost: 40
            minConnectionsPerHost: 1
            socketTimeout: 60000

这样好吗？

2019-12-10 21:45:32.862[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[           main][0;39m [36mc.p.test.Application     [0;39m [2m:[0;39m Started Application in 3.489 seconds (JVM running for 4.87)
[2m2019-12-10 21:46:31.977[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Closed connection [connectionIdlocalValue:3, serverValue:899] to localhost:27017 because it is past its maximum allowed idle time.
[2m2019-12-10 21:46:31.983[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionIdlocalValue:4, serverValue:903] to localhost:27017
[2m2019-12-10 21:47:28.917[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[ntLoopGroup-2-2][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionIdlocalValue:5, serverValue:905] to localhost:27017
[2m2019-12-10 21:47:31.122[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[ntLoopGroup-2-3][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionIdlocalValue:6, serverValue:906] to localhost:27017
[2m2019-12-10 21:47:31.974[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Closed connection [connectionIdlocalValue:4, serverValue:903] to localhost:27017 because it is past its maximum allowed idle time.
[2m2019-12-10 21:47:31.981[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionIdlocalValue:7, serverValue:907] to localhost:27017
[2m2019-12-10 21:48:31.974[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Closed connection [connectionIdlocalValue:7, serverValue:907] to localhost:27017 because it is past its maximum allowed idle time.
[2m2019-12-10 21:48:31.983[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionIdlocalValue:8, serverValue:910] to localhost:27017
[2m2019-12-10 21:49:31.975[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Closed connection [connectionIdlocalValue:8, serverValue:910] to localhost:27017 because it is past its maximum allowed idle time.
[2m2019-12-10 21:49:31.981[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Opened connection [connectionIdlocalValue:9, serverValue:913] to localhost:27017
[2m2019-12-10 21:50:31.974[0;39m [32m INFO[0;39m [35m20092[0;39m [2m---[0;39m [2m[imer-1-thread-1][0;39m [36morg.mongodb.driver.connection           [0;39m [2m:[0;39m Closed connection [connectionIdlocalValue:9, serverValue:913] to localhost:27017 because it is past its maximum allowed idle time.
[2m

在这方面有最佳实践吗？

进一步的编辑 正如 cmets 中所建议的，我使用了uri: mongodb://localhost:27017/LocalMongo?maxpoolsize=5 将最大池大小设置为 5。mongo 确保按预期最多打开 5 个连接，并且一旦达到空闲超时，连接就会过期。 但是，每次连接终止时，都会打开一个新连接。基本上，看起来它确保打开了至少 5 个连接。有没有我在这里遗漏的概念？

日志：

Opened connection [connectionIdlocalValue:2, serverValue:2843] to localhost:27017
Opened connection [connectionIdlocalValue:3, serverValue:2844] to localhost:27017
: Opened connection [connectionIdlocalValue:4, serverValue:2846] to localhost:27017
2019-12-13 13:00:17.316  INFO 10748 --- [ntLoopGroup-2-3] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:5, serverValue:2848] to localhost:27017
2019-12-13 13:00:20.436  INFO 10748 --- [ntLoopGroup-2-4] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:6, serverValue:2849] to localhost:27017
2019-12-13 13:00:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:3, serverValue:2844] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:00:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:7, serverValue:2851] to localhost:27017
2019-12-13 13:01:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:7, serverValue:2851] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:01:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:8, serverValue:2854] to localhost:27017
2019-12-13 13:02:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:8, serverValue:2854] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:02:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:9, serverValue:2856] to localhost:27017
2019-12-13 13:03:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:9, serverValue:2856] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:03:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:10, serverValue:2859] to localhost:27017
2019-12-13 13:04:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:10, serverValue:2859] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:04:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:11, serverValue:2862] to localhost:27017
2019-12-13 13:05:39.122  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:11, serverValue:2862] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:05:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:12, serverValue:2865] to localhost:27017
2019-12-13 13:06:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:12, serverValue:2865] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:06:39.126  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:13, serverValue:2868] to localhost:27017
2019-12-13 13:07:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:13, serverValue:2868] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:07:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:14, serverValue:2870] to localhost:27017
2019-12-13 13:08:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:14, serverValue:2870] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:08:39.128  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:15, serverValue:2873] to localhost:27017
2019-12-13 13:09:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:15, serverValue:2873] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:09:39.125  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:16, serverValue:2876] to localhost:27017
2019-12-13 13:10:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:16, serverValue:2876] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:10:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:17, serverValue:2879] to localhost:27017
2019-12-13 13:11:39.122  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:17, serverValue:2879] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:11:39.129  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:18, serverValue:2882] to localhost:27017
2019-12-13 13:12:39.123  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Closed connection [connectionIdlocalValue:18, serverValue:2882] to localhost:27017 because it is past its maximum allowed idle time.
2019-12-13 13:12:39.124  INFO 10748 --- [imer-1-thread-1] org.mongodb.driver.connection            : Opened connection [connectionIdlocalValue:19, serverValue:2885] to localhost:27017

如果需要更多信息，请告诉我。我将在此处编辑此空间。

【问题讨论】：

它似乎正在关闭所有以前打开的连接，不是吗？

不是全部...例如本地值为 1,2,3 的 connectionId 没有关闭...对于生产系统，每台主机的理想连接数是多少？

【参考方案1】：

这是由设置maxConnectionIdleTime=1引起的，您也可以在日志中看到：...because it is past its maximum allowed idle time.

【讨论】：

但是每个主机的最大连接数也是1，那么它是如何打开多个连接的呢？我在第二组日志之前也更改了配置。我将在编辑部分添加它们

如果您需要限制每个应用程序的连接数，请改用连接池设置。例如，只需像这样更改连接字符串：spring.data.mongodb.uri=mongodb://localhost:27017?maxIdleTimeMS=1500000&maxpoolsize=1

@ChiragC 以防万一它是如此简单 - 您粘贴在那里的配置将 minConnectionsPerHost 设置为 1，而不是 maxConnectionsPerHost。

@VladK 我使用您建议的参数将最大池大小设置为 5。现在，我在应用程序上添加了一些负载，它最终按预期打开了最多 5 个连接。一旦这些连接变得空闲，连接就会关闭，这也是预期的。但每次关闭一个连接时，它都会打开一个新连接，确保至少有 5 个连接。基本上， maxpoolsize=5 似乎更多的是要打开的最小连接数。我在这里错过了什么吗？

没有这样的概念，max pool只定义了上限。可能它是由 MongoClientOptions 以某种方式引起的。我建议调试 Spring Mongo AutoConfigs，有时这很棘手......我最终没有暴露 MongoClientOptions 并在连接字符串中拥有所有设置，检查类 com.mongodb.ConnectionString 以获取属性名称和描述。