Netty HTTP2 帧转发/代理 - 管道配置问题

Posted 2023-03-10

【中文标题】Netty HTTP2 帧转发/代理 - 管道配置问题

【英文标题】：Netty HTTP2 Frame Forwarding/Proxing - pipeline config question

【发布时间】：2019-09-20 02:00:21

【问题描述】：

我正在尝试创建一个 Netty (4.1) POC，它可以将 h2c（没有 TLS 的 HTTP2）帧转发到 h2c 服务器 - 即本质上是创建一个 Netty h2c 代理服务。 Wireshark 显示 Netty 将帧发送出去，h2c 服务器进行回复（例如使用响应标头和数据），尽管我在 Netty 本身中接收/处理响应 HTTP 帧时遇到了一些问题。

作为起点，我调整了 multiplex.server 示例 (io.netty.example.http2.helloworld.multiplex.server)，以便在 HelloWorldHttp2Handler 中连接到远程节点，而不是使用虚拟消息进行响应：

    @Override
    public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception 
    Channel remoteChannel = null;

    // create or retrieve the remote channel (one to one mapping) associated with this incoming (client) channel
    synchronized (lock) 
        if (!ctx.channel().hasAttr(remoteChannelKey)) 
            remoteChannel = this.connectToRemoteBlocking(ctx.channel());
            ctx.channel().attr(remoteChannelKey).set(remoteChannel);
         else 
            remoteChannel = ctx.channel().attr(remoteChannelKey).get();
        
    

    if (msg instanceof Http2HeadersFrame) 
        onHeadersRead(remoteChannel, (Http2HeadersFrame) msg);
     else if (msg instanceof Http2DataFrame) 
        final Http2DataFrame data = (Http2DataFrame) msg;
        onDataRead(remoteChannel, (Http2DataFrame) msg);
        send(ctx.channel(), new DefaultHttp2WindowUpdateFrame(data.initialFlowControlledBytes()).stream(data.stream()));
     else 
        super.channelRead(ctx, msg);
    


private void send(Channel remoteChannel, Http2Frame frame) 
    remoteChannel.writeAndFlush(frame).addListener(new GenericFutureListener() 
        @Override
        public void operationComplete(Future future) throws Exception 
            if (!future.isSuccess()) 
                future.cause().printStackTrace();
            
        
    );


/**
 * If receive a frame with end-of-stream set, send a pre-canned response.
 */
private void onDataRead(Channel remoteChannel, Http2DataFrame data) throws Exception 
    if (data.isEndStream()) 
        send(remoteChannel, data);
     else 
        // We do not send back the response to the remote-peer, so we need to release it.
        data.release();
    


/**
 * If receive a frame with end-of-stream set, send a pre-canned response.
 */
private void onHeadersRead(Channel remoteChannel, Http2HeadersFrame headers)
        throws Exception 
    if (headers.isEndStream()) 
        send(remoteChannel, headers);
    


private Channel connectToRemoteBlocking(Channel clientChannel) 
    try 
        Bootstrap b = new Bootstrap();
        b.group(new NioEventLoopGroup());
        b.channel(NiosocketChannel.class);
        b.option(ChannelOption.SO_KEEPALIVE, true);
        b.remoteAddress("localhost", H2C_SERVER_PORT);
        b.handler(new Http2ClientInitializer());

        final Channel channel = b.connect().syncUninterruptibly().channel();

        channel.config().setAutoRead(true);
        channel.attr(clientChannelKey).set(clientChannel);

        return channel;
     catch (Exception e) 
        e.printStackTrace();
        return null;
    

在初始化通道管道时（在Http2ClientInitializer），如果我这样做：

@Override
public void initChannel(SocketChannel ch) throws Exception 
    ch.pipeline().addLast(Http2MultiplexCodecBuilder.forClient(new Http2OutboundClientHandler()).frameLogger(TESTLOGGER).build());
    ch.pipeline().addLast(new UserEventLogger());

然后我可以看到在 Wireshark 中正确转发的帧，并且 h2c 服务器回复了标头和帧数据，但 Netty 回复了 GOAWAY [INTERNAL_ERROR]，原因是：

14:23:09.324 [nioEventLoopGroup-3-1] 警告 i.n.channel.DefaultChannelPipeline - exceptionCaught() 事件是 开火了，它到达了管道的尾部。它通常意味着 管道中的最后一个处理程序没有处理异常。 java.lang.IllegalStateException：需要流对象 标识符：1 在 io.netty.handler.codec.http2.Http2FrameCodec$FrameListener.requireStream(Http2FrameCodec.java:587) 在 io.netty.handler.codec.http2.Http2FrameCodec$FrameListener.onHeadersRead(Http2FrameCodec.java:550) 在 io.netty.handler.codec.http2.Http2FrameCodec$FrameListener.onHeadersRead(Http2FrameCodec.java:543)...

如果我改为尝试使其具有 http2 客户端示例中的管道配置，例如：

@Override
public void initChannel(SocketChannel ch) throws Exception 
    final Http2Connection connection = new DefaultHttp2Connection(false);

    ch.pipeline().addLast(
        new Http2ConnectionHandlerBuilder()
            .connection(connection)
            .frameLogger(TESTLOGGER)
            .frameListener(new DelegatingDecompressorFrameListener(connection, new InboundHttp2ToHttpAdapterBuilder(connection)
                .maxContentLength(maxContentLength)
                .propagateSettings(true)
                .build() ))
            .build());

然后我得到：

java.lang.UnsupportedOperationException：不支持的消息类型： DefaultHttp2HeadersFrame（预期：ByteBuf，FileRegion）在 io.netty.channel.nio.AbstractNioByteChannel.filterOutboundMessage(AbstractNioByteChannel.java:283) 在 io.netty.channel.AbstractChannel$AbstractUnsafe.write(AbstractChannel.java:882) 在 io.netty.channel.DefaultChannelPipeline$HeadContext.write(DefaultChannelPipeline.java:1365)

如果我再添加一个 HTTP2 帧编解码器（Http2MultiplexCodec 或 Http2FrameCodec）：

@Override
    public void initChannel(SocketChannel ch) throws Exception 
        final Http2Connection connection = new DefaultHttp2Connection(false);

        ch.pipeline().addLast(
            new Http2ConnectionHandlerBuilder()
                .connection(connection)
                .frameLogger(TESTLOGGER)
                .frameListener(new DelegatingDecompressorFrameListener(connection, new InboundHttp2ToHttpAdapterBuilder(connection)
                    .maxContentLength(maxContentLength)
                    .propagateSettings(true)
                    .build() ))
                .build());

        ch.pipeline().addLast(Http2MultiplexCodecBuilder.forClient(new Http2OutboundClientHandler()).frameLogger(TESTLOGGER).build());
    

然后Netty发送两个连接前言帧，导致h2c服务器以GOAWAY [PROTOCOL_ERROR]拒绝：

---

这就是我遇到问题的地方 - 即配置远程通道管道，以便它可以正确发送 Http2Frame 对象，但在收到响应时也会在 Netty 中接收/处理它们。

请问有人有什么想法/建议吗？

【问题讨论】：

Possibly related 讨论（尽管提问者的解决方案最终求助于发送完整的DefaultFullHttpRequest；他无法使框架方法正常工作）。 Norman Maurer's answer/comment 说这应该可以使用 Multiplex/Frame 编解码器来实现，这很好知道，但是目前没有这样的客户端示例，我无法让它完全工作（如上所述）。跨度>

【参考方案1】：

我最终得到了这个工作；以下 Github 问题包含一些有用的代码/信息：

Generating a Http2StreamChannel, from a Channel A Http2Client with Http2MultiplexCode

我需要进一步调查一些注意事项，尽管该方法的要点是您需要将您的频道包装在 Http2StreamChannel 中，这意味着我的 connectToRemoteBlocking() 方法最终为：

private Http2StreamChannel connectToRemoteBlocking(Channel clientChannel) 
        try 
            Bootstrap b = new Bootstrap();
            b.group(new NioEventLoopGroup()); // TODO reuse existing event loop
            b.channel(NioSocketChannel.class);
            b.option(ChannelOption.SO_KEEPALIVE, true);
            b.remoteAddress("localhost", H2C_SERVER_PORT);
            b.handler(new Http2ClientInitializer());

            final Channel channel = b.connect().syncUninterruptibly().channel();

            channel.config().setAutoRead(true);
            channel.attr(clientChannelKey).set(clientChannel);

            // TODO make more robust, see example at https://github.com/netty/netty/issues/8692
            final Http2StreamChannelBootstrap bs = new Http2StreamChannelBootstrap(channel);
            final Http2StreamChannel http2Stream = bs.open().syncUninterruptibly().get();
            http2Stream.attr(clientChannelKey).set(clientChannel);
            http2Stream.pipeline().addLast(new Http2OutboundClientHandler()); // will read: DefaultHttp2HeadersFrame, DefaultHttp2DataFrame

            return http2Stream;
         catch (Exception e) 
            e.printStackTrace();
            return null;
        
    

那么为了防止“Stream object required for identifier: 1”错误（本质上就是说：'这个（客户端）HTTP2请求是新的，那么为什么我们有这个特定的流？' - 因为我们隐式重用了来自最初收到的“服务器”请求的流对象），所以我们需要在转发数据时更改为使用远程通道的流：

private void onHeadersRead(Http2StreamChannel remoteChannel, Http2HeadersFrame headers) throws Exception 
        if (headers.isEndStream()) 
            headers.stream(remoteChannel.stream());
            send(remoteChannel, headers);
        
    

然后配置的通道入站处理程序（由于它的用法我称之为Http2OutboundClientHandler）将以正常方式接收传入的HTTP2帧：

@Sharable
public class Http2OutboundClientHandler extends SimpleChannelInboundHandler<Http2Frame> 

    @Override
    public void exceptionCaught(ChannelHandlerContext ctx, Throwable cause) throws Exception 
        super.exceptionCaught(ctx, cause);
        cause.printStackTrace();
        ctx.close();
    

    @Override
    public void channelRead0(ChannelHandlerContext ctx, Http2Frame msg) throws Exception 
        System.out.println("Http2OutboundClientHandler Http2Frame Type: " + msg.getClass().toString());