springboot RestTemplate post 两个tcp数据包 webserver报错

Posted 2023-03-09 wangjun5159

问题

springboot RestTemplate post请求外部接口时，报错误；

org.springframework.web.client.ResourceAccessException: I/O error on POST request for "http://ip:port": 
Unexpected end of file from server; nested exception is java.net.SocketException: Unexpected end of file from server

同样的请求内容使用postman则不报错，使用wireshark抓包后发现，RestTemplate 发送的http虽然未达到tcp MSS(默认的1460字节)，但仍然分为两个tcp数据包，

而postman使用了一个tcp数据包，我们知道，应用层是无法控制tcp数据包数量的。
默认的，resttemplate使用java.net.HttpURLConnection发送请求，

The default constructor uses java.net.HttpURLConnection to perform requests. You can switch to a different HTTP library with an implementation of ClientHttpRequestFactory. There is built-in support
for the following:
Apache HttpComponents
Netty
OkHttp

从源码得知，HttpURLConnection在发送http body之前会先发送header并且调用了flush方法，

org.springframework.http.client.SimpleBufferingClientHttpRequest
protected ClientHttpResponse executeInternal(HttpHeaders headers, byte[] bufferedOutput) throws IOException 
//将http body写入到输出流
	FileCopyUtils.copy(bufferedOutput, this.connection.getOutputStream());
		

这里的 this.connection，是jdk的内部实现sun.net.www.protocol.http.HttpURLConnection，每次调用getOutputStream()就会调用flush方法

sun.net.www.http.HttpClient
public void writeRequests(MessageHeader head,
                              PosterOutputStream pos) throws IOException 
        requests = head;
        //将http header写入到输出流中
        requests.print(serverOutput);
        poster = pos;
        if (poster != null)
            poster.writeTo(serverOutput);
         //这一行很重要   
        serverOutput.flush();
    

而将RestTemplate实现换为apache httpclient后，请求接口也是正常的，抓包后发现发送了一个tcp数据包。查源码发现，apache httpclient往流中写入header后并未flush，可能就是这个原因引起的，apache httpclient最终使用的org.apache.http.impl.conn.DefaultClientConnection发送请求。

public void sendRequestHeader(final HttpRequest request) throws HttpException, IOException 
        if (log.isDebugEnabled()) 
            log.debug("Sending request: " + request.getRequestLine());
        
        super.sendRequestHeader(request);
        if (headerLog.isDebugEnabled()) 
            headerLog.debug(">> " + request.getRequestLine().toString());
            final Header[] headers = request.getAllHeaders();
            for (final Header header : headers) 
                headerLog.debug(">> " + header.toString());
            
        
    

org.apache.http.impl.AbstractHttpClientConnection
public void sendRequestHeader(final HttpRequest request)
            throws HttpException, IOException 
        Args.notNull(request, "HTTP request");
        assertOpen();
        this.requestWriter.write(request);
        this.metrics.incrementRequestCount();
    

org.apache.http.impl.io.AbstractMessageWriter
 public void write(final T message) throws IOException, HttpException 
        Args.notNull(message, "HTTP message");
        //写完http header后，并未调用flush
        writeHeadLine(message);
        for (final HeaderIterator it = message.headerIterator(); it.hasNext(); ) 
            final Header header = it.nextHeader();
            this.sessionBuffer.writeLine
                (lineFormatter.formatHeader(this.lineBuf, header));
        
        this.lineBuf.clear();
        this.sessionBuffer.writeLine(this.lineBuf);
    

回顾flush方法

翻阅API，java.io.OutputStream

/**
     * Flushes this output stream and forces any buffered output bytes
     * to be written out. The general contract of <code>flush</code> is
     * that calling it is an indication that, if any bytes previously
     * written have been buffered by the implementation of the output
     * stream, such bytes should immediately be written to their
     * intended destination.
     * <p>
     * If the intended destination of this stream is an abstraction provided by
     * the underlying operating system, for example a file, then flushing the
     * stream guarantees only that bytes previously written to the stream are
     * passed to the operating system for writing; it does not guarantee that
     * they are actually written to a physical device such as a disk drive.
     * <p>
     * The <code>flush</code> method of <code>OutputStream</code> does nothing.
     *
     * @exception  IOException  if an I/O error occurs.
     */
    public void flush() throws IOException 
    

翻译过来就是强制将输出流中缓存的字节数据写入到目的地；如果目的地是一个文件，那么只能确保缓存字节写入到操作系统不能确保实际写入到磁盘；

解决

推测对方的web server实现有问题，不支持一个post请求两个tcp数据包，最终使用apache httpclient来作为RestTemplate的实现方式。

RestTemplate template = new RestTemplate(new HttpComponentsClientHttpRequestFactory());