java.net.SocketException: No buffer space available (maximum connections reached?): connect

Posted 2023-02-14 流云一号

        最近，有项目反馈调用第三方接口失败，查不到数据。现场用soapUI工具试了下，点执行按钮，一闪而过，接口没有任何返回值。一开始以为是第三方服务问题，后来查看日志，发现是自己的问题，报错如下：

Caused by: java.net.SocketException: No buffer space available (maximum connections reached?): connect
	at java.net.DualStackPlainSocketImpl.connect0(Native Method)
	at java.net.DualStackPlainSocketImpl.socketConnect(Unknown Source)
	at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
	at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
	at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
	at java.net.PlainSocketImpl.connect(Unknown Source)
	at java.net.SocksSocketImpl.connect(Unknown Source)
	at java.net.Socket.connect(Unknown Source)
	at sun.net.NetworkClient.doConnect(Unknown Source)
	at sun.net.www.http.HttpClient.openServer(Unknown Source)
	at sun.net.www.http.HttpClient.openServer(Unknown Source)
	at sun.net.www.http.HttpClient.<init>(Unknown Source)
	at sun.net.www.http.HttpClient.New(Unknown Source)
	at sun.net.www.http.HttpClient.New(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect0(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.plainConnect(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.connect(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(Unknown Source)
	at sun.net.www.protocol.http.HttpURLConnection.getOutputStream(Unknown Source)
	at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.setupWrappedStream(URLConnectionHTTPConduit.java:274)
	at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleHeadersTrustCaching(HTTPConduit.java:1343)
	at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.onFirstWrite(HTTPConduit.java:1304)
	at org.apache.cxf.transport.http.URLConnectionHTTPConduit$URLConnectionWrappedOutputStream.onFirstWrite(URLConnectionHTTPConduit.java:307)
	at org.apache.cxf.io.AbstractWrappedOutputStream.write(AbstractWrappedOutputStream.java:47)
	at org.apache.cxf.io.AbstractThresholdOutputStream.write(AbstractThresholdOutputStream.java:69)
	at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1356)
	... 116 common frames omitted

        Caused by: java.net.SocketException: No buffer space available (maximum connections reached?): connect，看样子是连接数太多，耗尽了某种资源。

        执行命令：netstat -ano （windows下查看当前所有的连接），发现大量的TCP连接，如图：

         打开任务管理器，查看PID为60204的进程，发现是个Java进程，如图：

        然后，逐一排查，发现有个Java应用，由于配置的xxljob服务地址不正确，连接失败又不断重试，如图：

         上述问题导致TCP连接过多，资源耗尽，其他Java应用无法建立TCP连接，接口调用自然报错。找到问题了，关掉有问题的Java应用，配置正确的xxljob服务地址，重启，问题解决。