07 netty粘包和半包问题的处理策略和自定义协议的要素

1 数据传输的经典问题

1-1 问题概述

粘包和半包问题的本质：实际开发中，程序员在应用层消息的分界无法在传输层得到支持，脱离应用层讨论粘包和半包毫无意义。

粘包问题

现象：发送 abc def，接收 abcdef

发生场景	说明
接受方的应用层缓冲区较大	接收方的应用层的 ByteBuf 设置较大，接受方的传输层的滑动窗口比较小
接受方的传输层的接受窗口较大	传输层中发送方 256 bytes 是一个完整报文，但由于接收方处理不及时且缓冲区足够大，这 256 bytes 字节就会缓冲在接收方的滑动窗口中，缓冲了多个报文就会粘包
发送方的传输层缓冲区	Nagle算法会造成粘包

if 有数据要发送：
   if 可用窗口大小 >= MSS and 可发送的数据 >= MSS：
        立刻发送MSS大小的数据
   else ：
       if 有未确认的数据：
            将数据放入缓存等待接收ACK （多次小批量的数据发送会被合并一起发送）
       else：
            立刻发送数据

Nagle 算法

• 即使发送一个字节，也需要加入 tcp 头和 ip 头，也就是总字节数会使用 41 bytes，非常不经济。因此为了提高网络利用率，tcp 希望尽可能发送足够大的数据，这就是 Nagle 算法产生的缘由
• 该算法是指发送端即使还有应该发送的数据，但如果这部分数据很少的话，则进行延迟发送
  • 如果 SO_SNDBUF 的数据达到 MSS，则需要发送
  • 如果 SO_SNDBUF 中含有 FIN（表示需要连接关闭）这时将剩余数据发送，再关闭
  • 如果 TCP_NODELAY = true，则需要发送
  • 已发送的数据都收到 ack 时，则需要发送
  • 上述条件不满足，但发生超时（一般为 200ms）则需要发送
  • 除上述情况，延迟发送

Delay ack 和 Nagle算法

消息边界问题需要考虑的情况以及常用解决策略

半包问题

现象：发送 abcdef，接收 abc def

原因：

发生场景	说明
应用层缓冲区	ByteBuf的大小无法一次性存放传输过来的数据
传输层发送窗口限制	接收方的窗口只剩了 128 bytes，发送方的报文大小是 256 bytes，这时放不下了，只能先发送前 128 bytes，等待 ack 后才能发送剩余部分，这就造成了半包
实际链路容量限制	发送的数据超过 MSS 限制后，会将数据切分发送，就会造成半包

MSS 是最大段长度（maximum segment size），它是 MTU 去除 tcp 头和 ip 头后剩余能够作为数据传输的字节数最大报文段长度

• 链路层对一次能够发送的最大数据有限制，这个限制称之为 MTU（maximum transmission unit），不同的链路设备的 MTU 值也有所不同，例如

  • 以太网的 MTU 是 1500，FDDI（光纤分布式数据接口）的 MTU 是 4352，本地回环地址的 MTU 是 65535 - 本地测试不走网卡

• TCP 在传递大量数据时，会按照 MSS 大小将数据进行分割发送

• MSS 的值在三次握手时通知对方自己 MSS 的值，然后在两者之间选择一个小值作为 MSS

1-2 解决方案

粘包与半包问题的解决需要我们对消息间的分隔进行，可以从以下几个思路入手

解决思路	特点	netty提供的入站hanlder	实际案例
采用短连接，发一个包建立一次连接，这样连接建立到连接断开之间就是消息的边界	通过连接的建立与断开分隔离消息，每次连接与建立都会耗费额外时间，效率太低，无法解决半包问题
每一条消息采用固定长度	提前约定每个消息的固定长度，数据包的大小不好把握，不够灵活，只适合特定场景	固定长度解码器FixedLengthFrameDecoder
约定固定字符作为分隔符，如果超出指定长度仍未出现分隔符，则抛出异常	某些消息内容中需要包含分隔符，不具有通用性	基于分隔符的解码器DelimiterBasedFrameDecoder和LineBasedFrameDecoder	redis协议是采用回车+换行作为分隔离符
约定用定长字节表示接下来数据的长度，每条消息分为 head 和 body，head 中包含 body 的长度	现代应用层传输协议中消息头部通常会有消息体的长度。	基于长度字段的解码器LengthFieldBasedFrameDecoder	http协议

• 几种解决思路中最常用的就是通过长度字段来进行消息的分割，现有的应用层协议的消息头中就有消息长度字段。

1-3 代码实例

服务端代码

• 通过更改TCP和Netty应用缓冲区的大小，可以看到半包和粘包现象

 // 设置服务端的TCP的接受缓冲区(receive buffer)为10个字节,不进行设置的则是由双方自动协商
serverBootstrap.option(ChannelOption.SO_RCVBUF,10);
 // 设置Bytebuf的大小为16字节，netty默认Bytebuf为1024字节
serverBootstrap.childOption(ChannelOption.RCVBUF_ALLOCATOR,new AdaptiveRecvByteBufAllocator(16,16,16));

package application.typical_problem;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class Server {
    static final Logger log = LoggerFactory.getLogger(Client.class);
    void start() {
        NioEventLoopGroup boss = new NioEventLoopGroup(1);
        NioEventLoopGroup worker = new NioEventLoopGroup();
        try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            // 设置服务端的TCP的接受缓冲区(receive buffer)为10个字节,不进行设置的则是由双方自动协商
            // serverBootstrap.option(ChannelOption.SO_RCVBUF,10);

            serverBootstrap.channel(NioServerSocketChannel.class);
            // 设置Bytebuf的大小为16字节，netty默认Bytebuf为1024字节
            // serverBootstrap.childOption(ChannelOption.RCVBUF_ALLOCATOR,new AdaptiveRecvByteBufAllocator(16,16,16));
            serverBootstrap.group(boss, worker);
            serverBootstrap.childHandler(new ChannelInitializer<SocketChannel>() {
                @Override
                protected void initChannel(SocketChannel ch) throws Exception {
                    ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));
                    ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {
                        @Override
                        public void channelActive(ChannelHandlerContext ctx) throws Exception {
                            log.debug("connected {}", ctx.channel());
                            super.channelActive(ctx);
                        }

                        @Override
                        public void channelInactive(ChannelHandlerContext ctx) throws Exception {
                            log.debug("disconnect {}", ctx.channel());
                            super.channelInactive(ctx);
                        }
                    });
                }
            });
            ChannelFuture channelFuture = serverBootstrap.bind(8080);
            log.debug("{} binding...", channelFuture.channel());
            channelFuture.sync();
            log.debug("{} bound...", channelFuture.channel());
            channelFuture.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            log.error("server error", e);
        } finally {
            boss.shutdownGracefully();
            worker.shutdownGracefully();
            log.debug("stoped");
        }
    }

    public static void main(String[] args) {
        new Server().start();
    }
}

短连接

短连接客户端实例：

package application.typical_problem;
import io.netty.bootstrap.Bootstrap;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

// 每调用一次send()相当于重新建立一次连接并发送数据
// 每次短连接发送一部分数据，显然效率低下
public class NianBaoClient {
    static final Logger log = LoggerFactory.getLogger(NianBaoClient.class);
    public static void main(String[] args) {
        for (int i = 0; i < 10; i++) {
            send();
        }
    }
    private static void send() {
        NioEventLoopGroup worker = new NioEventLoopGroup();
        try {
            Bootstrap bootstrap = new Bootstrap();
            bootstrap.channel(NioSocketChannel.class);
            bootstrap.group(worker);
            bootstrap.handler(new ChannelInitializer<SocketChannel>() {
                @Override
                protected void initChannel(SocketChannel ch) throws Exception {
                    log.debug("conneted...");
                    ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));
                    ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {
                        @Override
                        public void channelActive(ChannelHandlerContext ctx) throws Exception {
                            log.debug("sending...");
                            ByteBuf buffer = ctx.alloc().buffer();
                            buffer.writeBytes(new byte[]{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15});
                            ctx.writeAndFlush(buffer);
                            // 发完即关
                            ctx.close();
                        }
                    });
                }
            });
            ChannelFuture channelFuture = bootstrap.connect("localhost", 8080).sync();
            channelFuture.channel().closeFuture().sync();

        } catch (InterruptedException e) {
            log.error("client error", e);
        } finally {
            worker.shutdownGracefully();       // 关闭线程池
        }
    }
}

固定长度的编码器

• netty提供了固定长度解码器作为handler对消息进行切分

固定长度解码器的作用：

• 本质上是官方实现的一个handler

 * +---+----+------+----+
 * | A | BC | DEFG | HI |
 * +---+----+------+----+
  * +-----+-----+-----+
 * | ABC | DEF | GHI |
 * +-----+-----+-----+
 
该解码器将收到的字节流分割成固定长度的字节片段，如果我们收到了上述的four fragmented packets，那么经过这个特征的handler就可以得到三个长度相同的片段

通过在服务端添加固定长度分割的入站handler得到固定长度的消息

服务端代码

package application.typical_problem;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.FixedLengthFrameDecoder;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;


public class FixedServer {
    static final Logger log = LoggerFactory.getLogger(Client.class);
    void start() {
        NioEventLoopGroup boss = new NioEventLoopGroup(1);
        NioEventLoopGroup worker = new NioEventLoopGroup();
        try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.channel(NioServerSocketChannel.class);
            serverBootstrap.group(boss, worker);
            serverBootstrap.childHandler(new ChannelInitializer<SocketChannel>() {
                @Override
                protected void initChannel(SocketChannel ch) throws Exception {
                    // 添加handler，该handler实现对消息的固定长度分割，这里是8个字节为固定长度
                    ch.pipeline().addLast(new FixedLengthFrameDecoder(8));
                    ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));
                    ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {
                        @Override
                        public void channelActive(ChannelHandlerContext ctx) throws Exception {
                            log.debug("connected {}", ctx.channel());
                            super.channelActive(ctx);
                        }

                        @Override
                        public void channelInactive(ChannelHandlerContext ctx) throws Exception {
                            log.debug("disconnect {}", ctx.channel());
                            super.channelInactive(ctx);
                        }
                    });
                }
            });
            ChannelFuture channelFuture = serverBootstrap.bind(8080);
            log.debug("{} binding...", channelFuture.channel());
            channelFuture.sync();
            log.debug("{} bound...", channelFuture.channel());
            channelFuture.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            log.error("server error", e);
        } finally {
            boss.shutdownGracefully();
            worker.shutdownGracefully();
            log.debug("stoped");
        }
    }

    public static void main(String[] args) {
        new FixedServer().start();
    }
}

客户端代码

package application.typical_problem;
import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.util.Random;

public class FixedClient {
    static final Logger log = LoggerFactory.getLogger(FixedClient.class);

    public static void main(String[] args) {
        NioEventLoopGroup worker = new NioEventLoopGroup();
        try {
            Bootstrap bootstrap = new Bootstrap();
            bootstrap.channel(NioSocketChannel.class);
            bootstrap.group(worker);
            bootstrap.handler(new ChannelInitializer<SocketChannel>() {
                @Override
                protected void initChannel(SocketChannel ch) throws Exception {
                    log.debug("connetted...");
                    ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));
                    ch.pipeline().addLast(new ChannelInboundHandlerAdapter() {
                        @Override
                        public void channelActive(ChannelHandlerContext ctx) throws Exception {
                            log.debug("sending...");
                            // 发送内容随机的数据包
                            Random r = new Random();
                            char c = 'a';
                            ByteBuf buffer = ctx.alloc().buffer();
                            for (int i = 0; i < 10; i++) {
                                byte[] bytes = new byte[8];
                                for (int j = 0; j < r.nextInt(8); j++) {
                                    bytes[j] = (byte) c;
                                }
                                c++;
                                buffer.writeBytes(bytes);
                            }
                            ctx.writeAndFlush(buffer);
                        }
                    });
                }
            });
            ChannelFuture channelFuture = bootstrap.connect("localhost", 8080).sync();
            channelFuture.channel().closeFuture().sync();

        } catch (InterruptedException e) {
            log.error("client error", e);
        } finally {
            worker.shutdownGracefully();
        }
    }
}

客户端输出

16:09:20 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x054526db] REGISTERED
16:09:20 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x054526db] CONNECT: localhost/127.0.0.1:8080
16:09:20 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x054526db, L:/127.0.0.1:14833 - R:localhost/127.0.0.1:8080] ACTIVE
16:09:20 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x054526db, L:/127.0.0.1:14833 - R:localhost/127.0.0.1:8080] WRITE: 80B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 61 61 00 00 00 00 00 00 62 62 62 00 00 00 00 00 |aa......bbb.....|
|00000010| 00 00 00 00 00 00 00 00 64 64 64 64 00 00 00 00 |........dddd....|
|00000020| 00 00 00 00 00 00 00 00 66 66 66 00 00 00 00 00 |........fff.....|
|00000030| 67 00 00 00 00 00 00 00 68 00 00 00 00 00 00 00 |g.......h.......|
|00000040| 69 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 |i...............|
+--------+-------------------------------------------------+----------------+
16:09:20 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x054526db, L:/127.0.0.1:14833 - R:localhost/127.0.0.1:8080] FLUSH

服务端输出

16:09:20 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x9af7db67, L:/127.0.0.1:8080 - R:/127.0.0.1:14833] READ: 8B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 61 61 00 00 00 00 00 00                         |aa......        |
+--------+-------------------------------------------------+----------------+
16:09:20 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x9af7db67, L:/127.0.0.1:8080 - R:/127.0.0.1:14833] READ: 8B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 62 62 62 00 00 00 00 00                         |bbb.....        |
+--------+-------------------------------------------------+----------------+
16:09:20 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x9af7db67, L:/127.0.0.1:8080 - R:/127.0.0.1:14833] READ: 8B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 00 00 00 00 00                         |........        |
+--------+-------------------------------------------------+----------------+
16:09:20 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x9af7db67, L:/127.0.0.1:8080 - R:/127.0.0.1:14833] READ: 8B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 64 64 64 64 00 00 00 00                         |dddd....        |
+--------+-------------------------------------------------+----------------+
16:09:20 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x9af7db67, L:/127.0.0.1:8080 - R:/127.0.0.1:14833] READ: 8B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 00 00 00 00 00                         |........        |
+--------+-------------------------------------------------+----------------+
16:09:20 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x9af7db67, L:/127.0.0.1:8080 - R:/127.0.0.1:14833] READ: 8B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 66 66 66 00 00 00 00 00                         |fff.....        |
+--------+-------------------------------------------------+----------------+
16:09:20 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x9af7db67, L:/127.0.0.1:8080 - R:/127.0.0.1:14833] READ: 8B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 67 00 00 00 00 00 00 00                         |g.......        |
+--------+-------------------------------------------------+----------------+
16:09:20 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x9af7db67, L:/127.0.0.1:8080 - R:/127.0.0.1:14833] READ: 8B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 00 00 00 00 00 00 00                         |h.......        |
+--------+-------------------------------------------------+----------------+
16:09:20 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x9af7db67, L:/127.0.0.1:8080 - R:/127.0.0.1:14833] READ: 8B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 69 00 00 00 00 00 00 00                         |i.......        |
+--------+-------------------------------------------------+----------------+
16:09:20 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x9af7db67, L:/127.0.0.1:8080 - R:/127.0.0.1:14833] READ: 8B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 00 00 00 00 00 00 00 00                         |........        |
+--------+-------------------------------------------------+----------------+
16:09:20 [DEBUG] [nioEventLoopGroup-3-1] i.n.h.l.LoggingHandler - [id: 0x9af7db67, L:/127.0.0.1:8080 - R:/127.0.0.1:14833] READ COMPLETE

从输出可以看出，服务端确实是以8个字符为单位对消息体进行解码。

采用分割符的编码器

• netty提供了2个行解码器

LineBasedFrameDecoder:支持换行符进行分割，在windows(\r\n)和Linux平台(\n)都能够得到支持

public LineBasedFrameDecoder(final int maxLength) {
        this(maxLength, true, false);
}

DelimiterBasedFrameDecoder:支持自定义的分割符

public DelimiterBasedFrameDecoder(int maxFrameLength, ByteBuf delimiter) {
	this(maxFrameLength, true, delimiter);
}

• 采用分隔符作用消息的分界时，需要对消息的最长长度进行定义
  

基于长度字段的编码器

LengthFieldBasedFrameDecoder:根据长度字段对bytebuf进行划分

The value of the length field in this example is 12 (0x0C) which represents the length of "HELLO, WORLD". By default, the decoder assumes that the length field represents the number of the bytes that follows the length field. Therefore, it can be decoded with the simplistic parameter combination.

 lengthFieldOffset   = 0       // 0表示从头开始就是表示长度的字段
 lengthFieldLength   = 2       // 2表示长度字段占用两个字节
 lengthAdjustment    = 0
 initialBytesToStrip = 0 (= do not strip header)
  BEFORE DECODE (14 bytes)         AFTER DECODE (14 bytes)
 +--------+----------------+      +--------+----------------+
 | Length | Actual Content |----->| Length | Actual Content |
 | 0x000C | "HELLO, WORLD" |      | 0x000C | "HELLO, WORLD" |
 +--------+----------------+      +--------+----------------+
 ---------------------------------------------------------------------
 lengthFieldOffset   = 0
 lengthFieldLength   = 2
 lengthAdjustment    = 0
 initialBytesToStrip = 2 (= the length of the Length field)
 // initialBytesToStrip可以用于提取消息体信息，去除代表长度字段的两个字节

 BEFORE DECODE (14 bytes)         AFTER DECODE (12 bytes)
 +--------+----------------+      +----------------+
 | Length | Actual Content |----->| Actual Content |
 | 0x000C | "HELLO, WORLD" |      | "HELLO, WORLD" |
 +--------+----------------+      +----------------+
 ---------------------------------------------------------------------------
  lengthFieldOffset   =  0
 lengthFieldLength   =  2
 lengthAdjustment    = -2 (= the length of the Length field)
 initialBytesToStrip =  0

 BEFORE DECODE (14 bytes)         AFTER DECODE (14 bytes)
 +--------+----------------+      +--------+----------------+
 | Length | Actual Content |----->| Length | Actual Content |
 | 0x000E | "HELLO, WORLD" |      | 0x000E | "HELLO, WORLD" |
 +--------+----------------+      +--------+----------------+
 上述例子中长度字段是整个消息的字节数目，
 实际内容的字节数目 = 整个消息的字节数目 - 长度字段的字节数目 = Length -  lengthAdjustment
----------------------------------------------------------------------------
 lengthFieldOffset   = 2 (= the length of Header 1)
 lengthFieldLength   = 3
 lengthAdjustment    = 0
 initialBytesToStrip = 0

 BEFORE DECODE (17 bytes)                      AFTER DECODE (17 bytes)
 +----------+----------+----------------+      +----------+----------+----------------+
 | Header 1 |  Length  | Actual Content |----->| Header 1 |  Length  | Actual Content |
 |  0xCAFE  | 0x00000C | "HELLO, WORLD" |      |  0xCAFE  | 0x00000C | "HELLO, WORLD" |
 +----------+----------+----------------+      +----------+----------+----------------+

基于长度编码器的关键参数	含义
lengthFieldOffset	消息长度所在字段偏移量
lengthFieldLength	长度字段的长度（字节数）
lengthAdjustment	长度字段的位置作为基准，还有多少字节是内容，负数则是表明长度去除多少才是内容
initialBytesToStrip	从头剥离几个字节

基于长度的编码器使用实例：

package application.typical_problem;

import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufAllocator;
import io.netty.channel.embedded.EmbeddedChannel;
import io.netty.handler.codec.LengthFieldBasedFrameDecoder;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;

public class FieldBasedEncoder {
    public static void main(String[] args) {
        EmbeddedChannel ch = new EmbeddedChannel();
        // 消息长度，长度偏移量，长度字段的字节数，调整字节数，去除字节数
        ch.pipeline().addLast(new LengthFieldBasedFrameDecoder(1024,0,4,0,4));
        ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));
        ByteBuf bf = ByteBufAllocator.DEFAULT.buffer();
        send(bf,"hello,god");
        ch.writeInbound(bf);
    }

    public static void send(ByteBuf bf,String s){
        int n = s.length();
        byte[] arr = s.getBytes();
        bf.writeInt(n);         // 默认以大端模式写入int,写入长度
        // 注：如果希望长度字段后面再放入其他的字段比如版本号等可以通过lengthFieldOffset控制其他字段的长度
        bf.writeBytes(arr);
    }
}

实验结果

17:11:59 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ: 9B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 68 65 6c 6c 6f 2c 67 6f 64                      |hello,god       |
+--------+-------------------------------------------------+----------------+
17:11:59 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ COMPLETE

2 协议的设计与解析

2-1 Redis的通信协议

Redis协议英文说明

In RESP, the type of some data depends on the first byte:

type of data	first type
Simple Strings	the first byte of the reply is "+"
Errors	the reply is "-"
Integers	the first byte of the reply is ":"
Bulk Strings	the first byte of the reply is "$"
Arrays	the first byte of the reply is "*"

In RESP different parts of the protocol are always terminated with "\r\n" (CRLF).(回车和换行进行分隔)

适配器模式

Netty模拟Redis客户端发送命令： set name Amazing

package application.protocol_design;

import io.netty.bootstrap.Bootstrap;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufAllocator;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;

import java.nio.charset.Charset;

// 使用Netty模拟redis客户端发送set name Amazing命令
public class TestRedis {
    /*
       set name zhangsan
       $3set
       $4name
       $8Amazing
     */
    static final byte[] LINE = {13,10};   // 回车和换行字符的ascii码
    public static void main(String[] args) {
        NioEventLoopGroup g = new NioEventLoopGroup();
        try{
            Bootstrap bootstrap = new io.netty.bootstrap.Bootstrap();
            bootstrap.channel(NioSocketChannel.class);
            bootstrap.group(g);
            bootstrap.handler(new ChannelInitializer<SocketChannel>() {
                @Override
                public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
                    ByteBuf buf = (ByteBuf)msg;
                    System.out.println(buf.toString(Charset.defaultCharset()));
                }
                // 这里实现了向redis发送数组类型的数据为""*3\r\n$3\r\set\r\n$4\r\name\r\n$8\r\Amazing\r\n""
                // 该命令在redis服务端解析为 set name Amazing
                @Override
                protected void initChannel(SocketChannel ch) throws Exception {
                    ChannelPipeline p = ch.pipeline();
                    p.addLast(new ChannelInboundHandlerAdapter(){
                        @Override
                        public void channelActive(ChannelHandlerContext ctx) throws Exception {
                            ByteBuf bf = ByteBufAllocator.DEFAULT.buffer();
                            bf.writeBytes("*3".getBytes());
                            bf.writeBytes(LINE);
                            bf.writeBytes("$3".getBytes());
                            bf.writeBytes(LINE);
                            bf.writeBytes("set".getBytes());
                            bf.writeBytes(LINE);
                            bf.writeBytes("$4".getBytes());
                            bf.writeBytes(LINE);
                            bf.writeBytes("name".getBytes());
                            bf.writeBytes(LINE);
                            bf.writeBytes("$8".getBytes());
                            bf.writeBytes(LINE);
                            bf.writeBytes("Amazing".getBytes());
                            bf.writeBytes(LINE);
                            ctx.writeAndFlush(bf);
                        }
                    });

                }
            });
            ChannelFuture channelFuture = bootstrap.connect("localhost",6379).sync();
            channelFuture.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

2-2 Netty的Http协议编码解码器

public final class HttpServerCodec extends CombinedChannelDuplexHandler<HttpRequestDecoder, HttpResponseEncoder>
        implements HttpServerUpgradeHandler.SourceCodec

• HttpServerCodec是一个Http编码解码器，实现了入站和出站的handler的接口

模拟一个能够获取http报文信息的服务器：

package application.protocol_design;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.http.HttpServerCodec;
import io.netty.handler.codec.http2.InboundHttpToHttp2Adapter;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;



public class HttpServer {
    static final Logger log = LoggerFactory.getLogger(HttpServer.class);
    void start() {
        NioEventLoopGroup boss = new NioEventLoopGroup(1);
        try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.channel(NioServerSocketChannel.class);
            serverBootstrap.group(boss);
            serverBootstrap.childHandler(new ChannelInitializer<SocketChannel>() {
                @Override
                protected void initChannel(SocketChannel ch) throws Exception {
                    ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));
                    ch.pipeline().addLast(new HttpServerCodec());  // http协议的编码/解码器
                    ch.pipeline().addLast(new ChannelInboundHandlerAdapter(){
                        @Override
                        public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
                           log.debug("{}",msg.getClass());
                        }
                    });
                }
            });
            ChannelFuture channelFuture = serverBootstrap.bind(8080);
            channelFuture.sync();
            channelFuture.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            log.error("server error", e);
        } finally {
            boss.shutdownGracefully();
            log.debug("stoped");
        }
    }

    public static void main(String[] args) {
        new HttpServer().start();

    }
}

浏览器访问：http://localhost:8080/index

输出结果如下：

• 可以看到netty的HttpServerCodec()能够解析出采用http协议的文本信息

         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 47 45 54 20 2f 69 6e 64 65 78 20 48 54 54 50 2f |GET /index HTTP/|
|00000010| 31 2e 31 0d 0a 48 6f 73 74 3a 20 6c 6f 63 61 6c |1.1..Host: local|
|00000020| 68 6f 73 74 3a 38 30 38 30 0d 0a 43 6f 6e 6e 65 |host:8080..Conne|
|00000030| 63 74 69 6f 6e 3a 20 6b 65 65 70 2d 61 6c 69 76 |ction: keep-aliv|
|00000040| 65 0d 0a 43 61 63 68 65 2d 43 6f 6e 74 72 6f 6c |e..Cache-Control|
|00000050| 3a 20 6d 61 78 2d 61 67 65 3d 30 0d 0a 73 65 63 |: max-age=0..sec|
|00000060| 2d 63 68 2d 75 61 3a 20 22 47 6f 6f 67 6c 65 20 |-ch-ua: "Google |
|00000070| 43 68 72 6f 6d 65 22 3b 76 3d 22 39 35 22 2c 20 |Chrome";v="95", |
|00000080| 22 43 68 72 6f 6d 69 75 6d 22 3b 76 3d 22 39 35 |"Chromium";v="95|
|00000090| 22 2c 20 22 3b 4e 6f 74 20 41 20 42 72 61 6e 64 |", ";Not A Brand|
|000000a0| 22 3b 76 3d 22 39 39 22 0d 0a 73 65 63 2d 63 68 |";v="99"..sec-ch|
|000000b0| 2d 75 61 2d 6d 6f 62 69 6c 65 3a 20 3f 30 0d 0a |-ua-mobile: ?0..|
|000000c0| 73 65 63 2d 63 68 2d 75 61 2d 70 6c 61 74 66 6f |sec-ch-ua-platfo|
|000000d0| 72 6d 3a 20 22 57 69 6e 64 6f 77 73 22 0d 0a 55 |rm: "Windows"..U|
|000000e0| 70 67 72 61 64 65 2d 49 6e 73 65 63 75 72 65 2d |pgrade-Insecure-|
|000000f0| 52 65 71 75 65 73 74 73 3a 20 31 0d 0a 55 73 65 |Requests: 1..Use|
|00000100| 72 2d 41 67 65 6e 74 3a 20 4d 6f 7a 69 6c 6c 61 |r-Agent: Mozilla|
|00000110| 2f 35 2e 30 20 28 57 69 6e 64 6f 77 73 20 4e 54 |/5.0 (Windows NT|
|00000120| 20 31 30 2e 30 3b 20 57 69 6e 36 34 3b 20 78 36 | 10.0; Win64; x6|
|00000130| 34 29 20 41 70 70 6c 65 57 65 62 4b 69 74 2f 35 |4) AppleWebKit/5|
|00000140| 33 37 2e 33 36 20 28 4b 48 54 4d 4c 2c 20 6c 69 |37.36 (KHTML, li|
|00000150| 6b 65 20 47 65 63 6b 6f 29 20 43 68 72 6f 6d 65 |ke Gecko) Chrome|
|00000160| 2f 39 35 2e 30 2e 34 36 33 38 2e 36 39 20 53 61 |/95.0.4638.69 Sa|
|00000170| 66 61 72 69 2f 35 33 37 2e 33 36 0d 0a 41 63 63 |fari/537.36..Acc|
|00000180| 65 70 74 3a 20 74 65 78 74 2f 68 74 6d 6c 2c 61 |ept: text/html,a|
|00000190| 70 70 6c 69 63 61 74 69 6f 6e 2f 78 68 74 6d 6c |pplication/xhtml|
|000001a0| 2b 78 6d 6c 2c 61 70 70 6c 69 63 61 74 69 6f 6e |+xml,application|
|000001b0| 2f 78 6d 6c 3b 71 3d 30 2e 39 2c 69 6d 61 67 65 |/xml;q=0.9,image|
|000001c0| 2f 61 76 69 66 2c 69 6d 61 67 65 2f 77 65 62 70 |/avif,image/webp|
|000001d0| 2c 69 6d 61 67 65 2f 61 70 6e 67 2c 2a 2f 2a 3b |,image/apng,*/*;|
|000001e0| 71 3d 30 2e 38 2c 61 70 70 6c 69 63 61 74 69 6f |q=0.8,applicatio|
|000001f0| 6e 2f 73 69 67 6e 65 64 2d 65 78 63 68 61 6e 67 |n/signed-exchang|
|00000200| 65 3b 76 3d 62 33 3b 71 3d 30 2e 39 0d 0a 53 65 |e;v=b3;q=0.9..Se|
|00000210| 63 2d 46 65 74 63 68 2d 53 69 74 65 3a 20 6e 6f |c-Fetch-Site: no|
|00000220| 6e 65 0d 0a 53 65 63 2d 46 65 74 63 68 2d 4d 6f |ne..Sec-Fetch-Mo|
|00000230| 64 65 3a 20 6e 61 76 69 67 61 74 65 0d 0a 53 65 |de: navigate..Se|
|00000240| 63 2d 46 65 74 63 68 2d 55 73 65 72 3a 20 3f 31 |c-Fetch-User: ?1|
|00000250| 0d 0a 53 65 63 2d 46 65 74 63 68 2d 44 65 73 74 |..Sec-Fetch-Dest|
|00000260| 3a 20 64 6f 63 75 6d 65 6e 74 0d 0a 41 63 63 65 |: document..Acce|
|00000270| 70 74 2d 45 6e 63 6f 64 69 6e 67 3a 20 67 7a 69 |pt-Encoding: gzi|
|00000280| 70 2c 20 64 65 66 6c 61 74 65 2c 20 62 72 0d 0a |p, deflate, br..|
|00000290| 41 63 63 65 70 74 2d 4c 61 6e 67 75 61 67 65 3a |Accept-Language:|
|000002a0| 20 7a 68 2d 43 4e 2c 7a 68 3b 71 3d 30 2e 39 2c | zh-CN,zh;q=0.9,|
|000002b0| 65 6e 3b 71 3d 30 2e 38 2c 7a 68 2d 54 57 3b 71 |en;q=0.8,zh-TW;q|
|000002c0| 3d 30 2e 37 0d 0a 0d 0a                         |=0.7....        |
+--------+-------------------------------------------------+----------------+
23:39:10 [DEBUG] [nioEventLoopGroup-2-1] a.p.HttpServer - class io.netty.handler.codec.http.DefaultHttpRequest
Head
23:39:10 [DEBUG] [nioEventLoopGroup-2-1] a.p.HttpServer - class io.netty.handler.codec.http.LastHttpContent
body
23:39:10 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0xb501161e, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:12789] READ COMPLETE

从日志可以看出，channelRead方法被调用了两次，两次消息的类型不同，分别是：

• io.netty.handler.codec.http.DefaultHttpRequest
• io.netty.handler.codec.http.LastHttpContent

这两个类型分别代表http的请求头和请求体。

---

模拟一个能够获取http报文信息并反馈hello world的服务器：

• 注意该代码中通过

package application.protocol_design;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.*;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.SocketChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.handler.codec.http.*;
import io.netty.handler.logging.LogLevel;
import io.netty.handler.logging.LoggingHandler;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import static io.netty.handler.codec.http.HttpHeaderNames.CONTENT_LENGTH;


public class HttpServer {
    static final Logger log = LoggerFactory.getLogger(HttpServer.class);


    void start() {
        NioEventLoopGroup boss = new NioEventLoopGroup(1);
        try {
            ServerBootstrap serverBootstrap = new ServerBootstrap();
            serverBootstrap.channel(NioServerSocketChannel.class);
            serverBootstrap.group(boss);
            serverBootstrap.childHandler(new ChannelInitializer<SocketChannel>() {
                @Override
                protected void initChannel(SocketChannel ch) throws Exception {
                    ch.pipeline().addLast(new LoggingHandler(LogLevel.DEBUG));
                    ch.pipeline().addLast(new HttpServerCodec());  // http协议的编码/解码器
                    /*
                    ch.pipeline().addLast(new ChannelInboundHandlerAdapter(){
                        @Override
                        public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
                            log.debug("{}",msg.getClass());
                            if(msg instanceof HttpRequest) System.out.println("Head");
                            else System.out.println("body");
                        }
                    });*/
                    // SimpleChannelInboundHandler:能够处理特定类型的消息,消息类型与泛型一种
                    ch.pipeline().addLast(new SimpleChannelInboundHandler<HttpRequest>() {
                        @Override
                        protected void channelRead0(ChannelHandlerContext ctx, HttpRequest msg) throws Exception {
                            log.debug(msg.uri());
                            DefaultFullHttpResponse response = new DefaultFullHttpResponse(msg.protocolVersion(),
                                    HttpResponseStatus.OK);
                            byte[] bytes = "<h1>hello world</h1>".getBytes();
                            response.headers().setInt(CONTENT_LENGTH, bytes.length);
                            response.content().writeBytes(bytes);
                            ctx.writeAndFlush(response);
                        }
                    });
                }
            });
            ChannelFuture channelFuture = serverBootstrap.bind(8080);
            channelFuture.sync();
            channelFuture.channel().closeFuture().sync();
        } catch (InterruptedException e) {
            log.error("server error", e);
        } finally {
            boss.shutdownGracefully();
            log.debug("stoped");
        }
    }

    public static void main(String[] args) {
        new HttpServer().start();
    }
}

浏览器访问http://localhost:8080/index后的服务端日志输出

15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x07b5e461, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13010] REGISTERED
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x07b5e461, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13010] ACTIVE
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x5557ee13, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13011] REGISTERED
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x5557ee13, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13011] ACTIVE
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x5557ee13, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13011] READ: 686B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 47 45 54 20 2f 69 6e 64 65 78 20 48 54 54 50 2f |GET /index HTTP/|
|00000010| 31 2e 31 0d 0a 48 6f 73 74 3a 20 6c 6f 63 61 6c |1.1..Host: local|
|00000020| 68 6f 73 74 3a 38 30 38 30 0d 0a 43 6f 6e 6e 65 |host:8080..Conne|
|00000030| 63 74 69 6f 6e 3a 20 6b 65 65 70 2d 61 6c 69 76 |ction: keep-aliv|
|00000040| 65 0d 0a 73 65 63 2d 63 68 2d 75 61 3a 20 22 47 |e..sec-ch-ua: "G|
|00000050| 6f 6f 67 6c 65 20 43 68 72 6f 6d 65 22 3b 76 3d |oogle Chrome";v=|
|00000060| 22 39 35 22 2c 20 22 43 68 72 6f 6d 69 75 6d 22 |"95", "Chromium"|
|00000070| 3b 76 3d 22 39 35 22 2c 20 22 3b 4e 6f 74 20 41 |;v="95", ";Not A|
|00000080| 20 42 72 61 6e 64 22 3b 76 3d 22 39 39 22 0d 0a | Brand";v="99"..|
|00000090| 73 65 63 2d 63 68 2d 75 61 2d 6d 6f 62 69 6c 65 |sec-ch-ua-mobile|
|000000a0| 3a 20 3f 30 0d 0a 73 65 63 2d 63 68 2d 75 61 2d |: ?0..sec-ch-ua-|
|000000b0| 70 6c 61 74 66 6f 72 6d 3a 20 22 57 69 6e 64 6f |platform: "Windo|
|000000c0| 77 73 22 0d 0a 55 70 67 72 61 64 65 2d 49 6e 73 |ws"..Upgrade-Ins|
|000000d0| 65 63 75 72 65 2d 52 65 71 75 65 73 74 73 3a 20 |ecure-Requests: |
|000000e0| 31 0d 0a 55 73 65 72 2d 41 67 65 6e 74 3a 20 4d |1..User-Agent: M|
|000000f0| 6f 7a 69 6c 6c 61 2f 35 2e 30 20 28 57 69 6e 64 |ozilla/5.0 (Wind|
|00000100| 6f 77 73 20 4e 54 20 31 30 2e 30 3b 20 57 69 6e |ows NT 10.0; Win|
|00000110| 36 34 3b 20 78 36 34 29 20 41 70 70 6c 65 57 65 |64; x64) AppleWe|
|00000120| 62 4b 69 74 2f 35 33 37 2e 33 36 20 28 4b 48 54 |bKit/537.36 (KHT|
|00000130| 4d 4c 2c 20 6c 69 6b 65 20 47 65 63 6b 6f 29 20 |ML, like Gecko) |
|00000140| 43 68 72 6f 6d 65 2f 39 35 2e 30 2e 34 36 33 38 |Chrome/95.0.4638|
|00000150| 2e 36 39 20 53 61 66 61 72 69 2f 35 33 37 2e 33 |.69 Safari/537.3|
|00000160| 36 0d 0a 41 63 63 65 70 74 3a 20 74 65 78 74 2f |6..Accept: text/|
|00000170| 68 74 6d 6c 2c 61 70 70 6c 69 63 61 74 69 6f 6e |html,application|
|00000180| 2f 78 68 74 6d 6c 2b 78 6d 6c 2c 61 70 70 6c 69 |/xhtml+xml,appli|
|00000190| 63 61 74 69 6f 6e 2f 78 6d 6c 3b 71 3d 30 2e 39 |cation/xml;q=0.9|
|000001a0| 2c 69 6d 61 67 65 2f 61 76 69 66 2c 69 6d 61 67 |,image/avif,imag|
|000001b0| 65 2f 77 65 62 70 2c 69 6d 61 67 65 2f 61 70 6e |e/webp,image/apn|
|000001c0| 67 2c 2a 2f 2a 3b 71 3d 30 2e 38 2c 61 70 70 6c |g,*/*;q=0.8,appl|
|000001d0| 69 63 61 74 69 6f 6e 2f 73 69 67 6e 65 64 2d 65 |ication/signed-e|
|000001e0| 78 63 68 61 6e 67 65 3b 76 3d 62 33 3b 71 3d 30 |xchange;v=b3;q=0|
|000001f0| 2e 39 0d 0a 53 65 63 2d 46 65 74 63 68 2d 53 69 |.9..Sec-Fetch-Si|
|00000200| 74 65 3a 20 6e 6f 6e 65 0d 0a 53 65 63 2d 46 65 |te: none..Sec-Fe|
|00000210| 74 63 68 2d 4d 6f 64 65 3a 20 6e 61 76 69 67 61 |tch-Mode: naviga|
|00000220| 74 65 0d 0a 53 65 63 2d 46 65 74 63 68 2d 55 73 |te..Sec-Fetch-Us|
|00000230| 65 72 3a 20 3f 31 0d 0a 53 65 63 2d 46 65 74 63 |er: ?1..Sec-Fetc|
|00000240| 68 2d 44 65 73 74 3a 20 64 6f 63 75 6d 65 6e 74 |h-Dest: document|
|00000250| 0d 0a 41 63 63 65 70 74 2d 45 6e 63 6f 64 69 6e |..Accept-Encodin|
|00000260| 67 3a 20 67 7a 69 70 2c 20 64 65 66 6c 61 74 65 |g: gzip, deflate|
|00000270| 2c 20 62 72 0d 0a 41 63 63 65 70 74 2d 4c 61 6e |, br..Accept-Lan|
|00000280| 67 75 61 67 65 3a 20 7a 68 2d 43 4e 2c 7a 68 3b |guage: zh-CN,zh;|
|00000290| 71 3d 30 2e 39 2c 65 6e 3b 71 3d 30 2e 38 2c 7a |q=0.9,en;q=0.8,z|
|000002a0| 68 2d 54 57 3b 71 3d 30 2e 37 0d 0a 0d 0a       |h-TW;q=0.7....  |
+--------+-------------------------------------------------+----------------+
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] a.p.HttpServer - /index
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x5557ee13, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13011] WRITE: 59B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d |HTTP/1.1 200 OK.|
|00000010| 0a 63 6f 6e 74 65 6e 74 2d 6c 65 6e 67 74 68 3a |.content-length:|
|00000020| 20 32 30 0d 0a 0d 0a 3c 68 31 3e 68 65 6c 6c 6f | 20....<h1>hello|
|00000030| 20 77 6f 72 6c 64 3c 2f 68 31 3e                | world</h1>     |
+--------+-------------------------------------------------+----------------+
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x5557ee13, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13011] FLUSH
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x5557ee13, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13011] READ COMPLETE
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x5557ee13, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13011] READ: 612B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 47 45 54 20 2f 66 61 76 69 63 6f 6e 2e 69 63 6f |GET /favicon.ico|
|00000010| 20 48 54 54 50 2f 31 2e 31 0d 0a 48 6f 73 74 3a | HTTP/1.1..Host:|
|00000020| 20 6c 6f 63 61 6c 68 6f 73 74 3a 38 30 38 30 0d | localhost:8080.|
|00000030| 0a 43 6f 6e 6e 65 63 74 69 6f 6e 3a 20 6b 65 65 |.Connection: kee|
|00000040| 70 2d 61 6c 69 76 65 0d 0a 73 65 63 2d 63 68 2d |p-alive..sec-ch-|
|00000050| 75 61 3a 20 22 47 6f 6f 67 6c 65 20 43 68 72 6f |ua: "Google Chro|
|00000060| 6d 65 22 3b 76 3d 22 39 35 22 2c 20 22 43 68 72 |me";v="95", "Chr|
|00000070| 6f 6d 69 75 6d 22 3b 76 3d 22 39 35 22 2c 20 22 |omium";v="95", "|
|00000080| 3b 4e 6f 74 20 41 20 42 72 61 6e 64 22 3b 76 3d |;Not A Brand";v=|
|00000090| 22 39 39 22 0d 0a 73 65 63 2d 63 68 2d 75 61 2d |"99"..sec-ch-ua-|
|000000a0| 6d 6f 62 69 6c 65 3a 20 3f 30 0d 0a 55 73 65 72 |mobile: ?0..User|
|000000b0| 2d 41 67 65 6e 74 3a 20 4d 6f 7a 69 6c 6c 61 2f |-Agent: Mozilla/|
|000000c0| 35 2e 30 20 28 57 69 6e 64 6f 77 73 20 4e 54 20 |5.0 (Windows NT |
|000000d0| 31 30 2e 30 3b 20 57 69 6e 36 34 3b 20 78 36 34 |10.0; Win64; x64|
|000000e0| 29 20 41 70 70 6c 65 57 65 62 4b 69 74 2f 35 33 |) AppleWebKit/53|
|000000f0| 37 2e 33 36 20 28 4b 48 54 4d 4c 2c 20 6c 69 6b |7.36 (KHTML, lik|
|00000100| 65 20 47 65 63 6b 6f 29 20 43 68 72 6f 6d 65 2f |e Gecko) Chrome/|
|00000110| 39 35 2e 30 2e 34 36 33 38 2e 36 39 20 53 61 66 |95.0.4638.69 Saf|
|00000120| 61 72 69 2f 35 33 37 2e 33 36 0d 0a 73 65 63 2d |ari/537.36..sec-|
|00000130| 63 68 2d 75 61 2d 70 6c 61 74 66 6f 72 6d 3a 20 |ch-ua-platform: |
|00000140| 22 57 69 6e 64 6f 77 73 22 0d 0a 41 63 63 65 70 |"Windows"..Accep|
|00000150| 74 3a 20 69 6d 61 67 65 2f 61 76 69 66 2c 69 6d |t: image/avif,im|
|00000160| 61 67 65 2f 77 65 62 70 2c 69 6d 61 67 65 2f 61 |age/webp,image/a|
|00000170| 70 6e 67 2c 69 6d 61 67 65 2f 73 76 67 2b 78 6d |png,image/svg+xm|
|00000180| 6c 2c 69 6d 61 67 65 2f 2a 2c 2a 2f 2a 3b 71 3d |l,image/*,*/*;q=|
|00000190| 30 2e 38 0d 0a 53 65 63 2d 46 65 74 63 68 2d 53 |0.8..Sec-Fetch-S|
|000001a0| 69 74 65 3a 20 73 61 6d 65 2d 6f 72 69 67 69 6e |ite: same-origin|
|000001b0| 0d 0a 53 65 63 2d 46 65 74 63 68 2d 4d 6f 64 65 |..Sec-Fetch-Mode|
|000001c0| 3a 20 6e 6f 2d 63 6f 72 73 0d 0a 53 65 63 2d 46 |: no-cors..Sec-F|
|000001d0| 65 74 63 68 2d 44 65 73 74 3a 20 69 6d 61 67 65 |etch-Dest: image|
|000001e0| 0d 0a 52 65 66 65 72 65 72 3a 20 68 74 74 70 3a |..Referer: http:|
|000001f0| 2f 2f 6c 6f 63 61 6c 68 6f 73 74 3a 38 30 38 30 |//localhost:8080|
|00000200| 2f 69 6e 64 65 78 0d 0a 41 63 63 65 70 74 2d 45 |/index..Accept-E|
|00000210| 6e 63 6f 64 69 6e 67 3a 20 67 7a 69 70 2c 20 64 |ncoding: gzip, d|
|00000220| 65 66 6c 61 74 65 2c 20 62 72 0d 0a 41 63 63 65 |eflate, br..Acce|
|00000230| 70 74 2d 4c 61 6e 67 75 61 67 65 3a 20 7a 68 2d |pt-Language: zh-|
|00000240| 43 4e 2c 7a 68 3b 71 3d 30 2e 39 2c 65 6e 3b 71 |CN,zh;q=0.9,en;q|
|00000250| 3d 30 2e 38 2c 7a 68 2d 54 57 3b 71 3d 30 2e 37 |=0.8,zh-TW;q=0.7|
|00000260| 0d 0a 0d 0a                                     |....            |
+--------+-------------------------------------------------+----------------+
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] a.p.HttpServer - /favicon.ico
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x5557ee13, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13011] WRITE: 59B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d |HTTP/1.1 200 OK.|
|00000010| 0a 63 6f 6e 74 65 6e 74 2d 6c 65 6e 67 74 68 3a |.content-length:|
|00000020| 20 32 30 0d 0a 0d 0a 3c 68 31 3e 68 65 6c 6c 6f | 20....<h1>hello|
|00000030| 20 77 6f 72 6c 64 3c 2f 68 31 3e                | world</h1>     |
+--------+-------------------------------------------------+----------------+
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x5557ee13, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13011] FLUSH
15:16:17 [DEBUG] [nioEventLoopGroup-2-1] i.n.h.l.LoggingHandler - [id: 0x5557ee13, L:/0:0:0:0:0:0:0:1:8080 - R:/0:0:0:0:0:0:0:1:13011] READ COMPLETE

从日志信息可以看出浏览器和服务端通过http协议的交互流程如下：

主体	行为
浏览器	GET /index
客户端	HTTP/1.1 200 OK .content-length: 20.... hello world
浏览器	GET /favicon.ico
客户端	HTTP/1.1 200 OK .content-length: 20.... hello world

• 第二次浏览器自己主动请求GET网站的图标

2-3 自定义协议

常规的自定义协议包含元素

• 魔数，用来在第一时间判定是否是无效数据包

JVM字节码文件中首部四个字节也是魔数用于标识当前文件类型

• 版本号，可以支持协议的升级
• 序列化算法，消息正文到底采用哪种序列化反序列化方式，可以由此扩展，例如：json、protobuf、hessian、jdk

protobuf、hessian是二进制的序列化算法

• 指令类型，是登录、注册、单聊、群聊... 跟业务相关
• 请求序号，为了双工通信，提供异步能力
• 正文长度
• 消息正文

---

自定义消息类

根据上述要求可以定义为自己协议定义消息类如下：

• 消息类包含：序列ID，消息类型，存储<key,value>为<消息标识，消息的class对象>的HashMap

静态代码块在字节码层面的体现

该消息类在创建的时候，通过静态代码块在class对象加载后初始化HashMap

package application.protocol_design.message;

import lombok.Data;

import java.io.Serializable;
import java.util.HashMap;
import java.util.Map;

@Data
public abstract class Message implements Serializable {

    /**
     * 根据消息类型字节，获得对应的消息 class
     * @param messageType 消息类型字节
     * @return 消息 class
     */
    public static Class<? extends Message> getMessageClass(int messageType) {
        return messageClasses.get(messageType);
    }

    private int sequenceId;
    private int messageType;

    public abstract int getMessageType();

    public static final int LoginRequestMessage = 0;
    public static final int LoginResponseMessage = 1;
    public static final int ChatRequestMessage = 2;
    public static final int ChatResponseMessage = 3;
    public static final int GroupCreateRequestMessage = 4;
    public static final int GroupCreateResponseMessage = 5;
    public static final int GroupJoinRequestMessage = 6;
    public static final int GroupJoinResponseMessage = 7;
    public static final int GroupQuitRequestMessage = 8;
    public static final int GroupQuitResponseMessage = 9;
    public static final int GroupChatRequestMessage = 10;
    public static final int GroupChatResponseMessage = 11;
    public static final int GroupMembersRequestMessage = 12;
    public static final int GroupMembersResponseMessage = 13;
    public static final int PingMessage = 14;
    public static final int PongMessage = 15;
    /**
     * 请求类型 byte 值
     */
    public static final int RPC_MESSAGE_TYPE_REQUEST = 101;
    /**
     * 响应类型 byte 值
     */
    public static final int  RPC_MESSAGE_TYPE_RESPONSE = 102;

    private static final Map<Integer, Class<? extends Message>> messageClasses = new HashMap<>();
    static {
        messageClasses.put(LoginRequestMessage, LoginRequestMessage.class);
        messageClasses.put(LoginResponseMessage, LoginResponseMessage.class);
        messageClasses.put(ChatRequestMessage, ChatRequestMessage.class);
        messageClasses.put(ChatResponseMessage, ChatResponseMessage.class);
        messageClasses.put(GroupCreateRequestMessage, GroupCreateRequestMessage.class);
        messageClasses.put(GroupCreateResponseMessage, GroupCreateResponseMessage.class);
        messageClasses.put(GroupJoinRequestMessage, GroupJoinRequestMessage.class);
        messageClasses.put(GroupJoinResponseMessage, GroupJoinResponseMessage.class);
        messageClasses.put(GroupQuitRequestMessage, GroupQuitRequestMessage.class);
        messageClasses.put(GroupQuitResponseMessage, GroupQuitResponseMessage.class);
        messageClasses.put(GroupChatRequestMessage, GroupChatRequestMessage.class);
        messageClasses.put(GroupChatResponseMessage, GroupChatResponseMessage.class);
        messageClasses.put(GroupMembersRequestMessage, GroupMembersRequestMessage.class);
        messageClasses.put(GroupMembersResponseMessage, GroupMembersResponseMessage.class);
        messageClasses.put(RPC_MESSAGE_TYPE_REQUEST, RpcRequestMessage.class);
        messageClasses.put(RPC_MESSAGE_TYPE_RESPONSE, RpcResponseMessage.class);
    }
}

自定义协议的消息编码和解码器(handler)

• 在netty中通过继承ByteToMessageCodec类并重写相关方法实现自定义的编码/解码handler

package application.protocol_design.protocol;

import application.protocol_design.message.Message;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.ByteToMessageCodec;
import lombok.extern.slf4j.Slf4j;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.util.List;

@Slf4j
public class MessageCodec extends ByteToMessageCodec<Message> {
    static byte[] magicNum = {'l','u','c','k'};

    // 编码目标:将消息msg变化为遵循协议的字节数组放入到ByteBuf out
    @Override
    protected void encode(ChannelHandlerContext ctx, Message msg, ByteBuf out) throws Exception {
        Byte version = 1;            // 1字节的协议版本
        Byte serialWay = 0;          // 1字节的序列化方式: 0表示JDK,1表示json

        // 总字节数目 = 16(如果不是2的幂可以填充)
        out.writeBytes(magicNum);             // 4字节的协议魔数
        out.writeByte(version);                // 1字节的协议版本
        out.writeByte(serialWay);              // 1字节的序列化方式: 0表示JDK,1表示json
        out.writeByte(msg.getMessageType());   //  1字节指令类型
        out.writeInt(msg.getSequenceId());     // 4字节序列号

        //objectOutputStream:把对象转成字节数据的输出到文件中保存，对象的输出过程称为序列化，可实现对象的持久存储
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(bos);
        oos.writeObject(msg);
        byte[] content = bos.toByteArray();

        out.writeInt(content.length);              // 写入对象序列化的后的字节数组长度
        out.writeByte(0xff);                       //  填充字符，凑满2的幂为16
        out.writeBytes(content);                   // 写入对象序列化数组
    }

    // 解码目标:将字节数组转化为对象放入到List<Object> out
    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
        int magicNum = in.readInt();
        byte version = in.readByte();
        byte serialType = in.readByte();

        byte messageType = in.readByte();
        int sequenceId = in.readInt();

        int length = in.readInt();
        byte padding = in.readByte();
        byte[] arr = new byte[length];
        in.readBytes(arr,0,length);

        // 这里可以加入根据序列化协议调用不同序列化工具的类
        ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(arr));
        Message message = (Message)ois.readObject();
        out.add(message);
        log.debug("魔数：{} 版本：{} 序列化方式：{} 消息类型：{} 序列ID：长度：{} {}",magicNum,version,serialType,messageType,sequenceId,length);
        log.debug("{}",message);
    }
}

上述代码中的自定义协议的消息组成如下：

名称	魔数	协议版本	序列化协议	消息类型	序列ID	内容长度	填充字符	内容
所占字节数	4	1	1	1	4	4	1

• 填充字符让消息头大小变为为16字节，恰好是2的幂，比较规整。

• 上述代码中encode方法是将类按照自定义协议转化为byte数组，decode方法是将byte数组中提取类的实例对象。

自定义协议测试

package application.protocol_design.protocol;

import application.protocol_design.message.LoginRequestMessage;
import application.typical_problem.FieldBasedEncoder;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufAllocator;
import io.netty.channel.embedded.EmbeddedChannel;
import io.netty.handler.codec.LengthFieldBasedFrameDecoder;
import io.netty.handler.logging.LoggingHandler;

// 测试编码/解码的handler
public class TestMessageCodec {
    public static void main(String[] args) throws Exception {
        EmbeddedChannel ch = new EmbeddedChannel();
        // 打印日志的handler
        ch.pipeline().addLast(new LoggingHandler());
        // 基于长度的handler解决传输过程中的粘包和半包问题
        ch.pipeline().addLast(new LengthFieldBasedFrameDecoder(1024,11,4,1,0));
        // 自定义的协议的编码/解码器handler
        ch.pipeline().addLast(new MessageCodec());

        // 测试出站编码

        System.out.println("\n测试自定义协议的编码\n");


        LoginRequestMessage m = new LoginRequestMessage("god","dog");
        ch.writeOutbound(m);

        System.out.println("\n测试自定义协议的解码\n");
        // 测试入站解码
        ByteBuf bf = ByteBufAllocator.DEFAULT.buffer();
        new MessageCodec().encode(null,m,bf);
        // 这里将一条消息分两次发送，如果没有基于长度字段的编码器LengthFieldBasedFrameDecoder,那么会报错
        ByteBuf b1 = bf.slice(0,100);
        ByteBuf b2 = bf.slice(100,bf.readableBytes()-100); b1.retain();
        ch.writeInbound(b1); ch.writeInbound(b2);
    }
}

测试日志

测试自定义协议的编码

20:52:12 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] WRITE: 245B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 6c 75 63 6b 01 00 00 00 00 00 00 00 00 00 e5 ff |luck............|
|00000010| ac ed 00 05 73 72 00 37 61 70 70 6c 69 63 61 74 |....sr.7applicat|
|00000020| 69 6f 6e 2e 70 72 6f 74 6f 63 6f 6c 5f 64 65 73 |ion.protocol_des|
|00000030| 69 67 6e 2e 6d 65 73 73 61 67 65 2e 4c 6f 67 69 |ign.message.Logi|
|00000040| 6e 52 65 71 75 65 73 74 4d 65 73 73 61 67 65 ca |nRequestMessage.|
|00000050| ab b7 04 6f 14 ef a6 02 00 02 4c 00 08 70 61 73 |...o......L..pas|
|00000060| 73 77 6f 72 64 74 00 12 4c 6a 61 76 61 2f 6c 61 |swordt..Ljava/la|
|00000070| 6e 67 2f 53 74 72 69 6e 67 3b 4c 00 08 75 73 65 |ng/String;L..use|
|00000080| 72 6e 61 6d 65 71 00 7e 00 01 78 72 00 2b 61 70 |rnameq.~..xr.+ap|
|00000090| 70 6c 69 63 61 74 69 6f 6e 2e 70 72 6f 74 6f 63 |plication.protoc|
|000000a0| 6f 6c 5f 64 65 73 69 67 6e 2e 6d 65 73 73 61 67 |ol_design.messag|
|000000b0| 65 2e 4d 65 73 73 61 67 65 c9 91 c8 73 58 b1 e2 |e.Message...sX..|
|000000c0| 08 02 00 02 49 00 0b 6d 65 73 73 61 67 65 54 79 |....I..messageTy|
|000000d0| 70 65 49 00 0a 73 65 71 75 65 6e 63 65 49 64 78 |peI..sequenceIdx|
|000000e0| 70 00 00 00 00 00 00 00 00 74 00 03 64 6f 67 74 |p........t..dogt|
|000000f0| 00 03 67 6f 64                                  |..god           |
+--------+-------------------------------------------------+----------------+
20:52:12 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] FLUSH

测试自定义协议的解码

20:52:12 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ: 100B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 6c 75 63 6b 01 00 00 00 00 00 00 00 00 00 e5 ff |luck............|
|00000010| ac ed 00 05 73 72 00 37 61 70 70 6c 69 63 61 74 |....sr.7applicat|
|00000020| 69 6f 6e 2e 70 72 6f 74 6f 63 6f 6c 5f 64 65 73 |ion.protocol_des|
|00000030| 69 67 6e 2e 6d 65 73 73 61 67 65 2e 4c 6f 67 69 |ign.message.Logi|
|00000040| 6e 52 65 71 75 65 73 74 4d 65 73 73 61 67 65 ca |nRequestMessage.|
|00000050| ab b7 04 6f 14 ef a6 02 00 02 4c 00 08 70 61 73 |...o......L..pas|
|00000060| 73 77 6f 72                                     |swor            |
+--------+-------------------------------------------------+----------------+
20:52:12 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ COMPLETE
20:52:12 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ: 145B
         +-------------------------------------------------+
         |  0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f |
+--------+-------------------------------------------------+----------------+
|00000000| 64 74 00 12 4c 6a 61 76 61 2f 6c 61 6e 67 2f 53 |dt..Ljava/lang/S|
|00000010| 74 72 69 6e 67 3b 4c 00 08 75 73 65 72 6e 61 6d |tring;L..usernam|
|00000020| 65 71 00 7e 00 01 78 72 00 2b 61 70 70 6c 69 63 |eq.~..xr.+applic|
|00000030| 61 74 69 6f 6e 2e 70 72 6f 74 6f 63 6f 6c 5f 64 |ation.protocol_d|
|00000040| 65 73 69 67 6e 2e 6d 65 73 73 61 67 65 2e 4d 65 |esign.message.Me|
|00000050| 73 73 61 67 65 c9 91 c8 73 58 b1 e2 08 02 00 02 |ssage...sX......|
|00000060| 49 00 0b 6d 65 73 73 61 67 65 54 79 70 65 49 00 |I..messageTypeI.|
|00000070| 0a 73 65 71 75 65 6e 63 65 49 64 78 70 00 00 00 |.sequenceIdxp...|
|00000080| 00 00 00 00 00 74 00 03 64 6f 67 74 00 03 67 6f |.....t..dogt..go|
|00000090| 64                                              |d               |
+--------+-------------------------------------------------+----------------+
20:52:12 [DEBUG] [main] a.p.p.MessageCodec - 魔数：1819632491 版本：1 序列化方式：0 消息类型：0 序列ID：长度：0 229
20:52:12 [DEBUG] [main] a.p.p.MessageCodec - LoginRequestMessage(super=Message(sequenceId=0, messageType=0), username=god, password=dog)
20:52:12 [DEBUG] [main] i.n.h.l.LoggingHandler - [id: 0xembedded, L:embedded - R:embedded] READ COMPLETE

Process finished with exit code 0

// 基于长度的handler解决传输过程中的粘包和半包问题
ch.pipeline().addLast(new LengthFieldBasedFrameDecoder(1024,11,4,1,0));
// 自定义的协议的编码/解码器handler
ch.pipeline().addLast(new MessageCodec());

• 代码中LengthFieldBasedFrameDecoder和MessageCodec这两个handler形成配合，通过Netty的基于字段的编码器（handler）解决粘包和半包问题，所有数据经过LengthFieldBasedFrameDecoder会得到一个完整消息的byte数组，然后再按照MessageCodec中自定义协议解码

2-4 handler实例共享问题

问题：channel实例中数据流的处理需要通过handler实例实现，多个channel实例能够共享相同的handler实例？

策略：需要具体问题具体分析，基本原则是无状态的handler实例可以共享使用，netty官方提供的handler中可以通过@Sharable注解判别该handler是否能够被多个channel共享使用。

• 比如LoggingHandler提供日志输出功能，官方源码中有@Sharable注解，因此该handler能够被多个channel使用。

@Sharable
@SuppressWarnings({ "StringConcatenationInsideStringBufferAppend", "StringBufferReplaceableByString" })
public class LoggingHandler extends ChannelDuplexHandler {

    private static final LogLevel DEFAULT_LEVEL = LogLevel.DEBUG;

    protected final InternalLogger logger;
    protected final InternalLogLevel internalLevel;

    private final LogLevel level;

    /**
     * Creates a new instance whose logger name is the fully qualified class
     * name of the instance with hex dump enabled.
     */
    public LoggingHandler() {
        this(DEFAULT_LEVEL);
    }
...

• 比如LengthFieldBasedFrameDecoder这个解码器handler的单个实例不能够被多个channel共享使用。

ublic class LengthFieldBasedFrameDecoder extends ByteToMessageDecoder {

    private final ByteOrder byteOrder;
    private final int maxFrameLength;
    private final int lengthFieldOffset;
    private final int lengthFieldLength;
    private final int lengthFieldEndOffset;
    private final int lengthAdjustment;
    private final int initialBytesToStrip;
    private final boolean failFast;
    private boolean discardingTooLongFrame;
    private long tooLongFrameLength;
    private long bytesToDiscard;

---

需求：将自定义的编解码器定义为可共享的handler供其他人使用

• 方法：这里没有继承2-3中ByteToMessageCodec，官方约定该类的子类无法加上@Sharable注解，继承MessageToMessageCodec，这样就能够加上@Sharable注解

package application.protocol_design.protocol;

import application.protocol_design.message.Message;
import io.netty.buffer.ByteBuf;
import io.netty.channel.ChannelHandler;
import io.netty.channel.ChannelHandlerContext;
import io.netty.handler.codec.MessageToMessageCodec;
import lombok.extern.slf4j.Slf4j;

import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.util.List;
/*
    该处理的前置handler必须是LengthFieldBasedDecoder
 */
@Slf4j
@ChannelHandler.Sharable
public class MessageCodecSharable extends MessageToMessageCodec<ByteBuf, Message> {
    static byte[] magicNum = {'l','u','c','k'};
    @Override
    protected void encode(ChannelHandlerContext ctx, Message msg, List<Object> outList) throws Exception {
        ByteBuf out = ctx.alloc().buffer();
        Byte version = 1;            // 1字节的协议版本
        Byte serialWay = 0;          // 1字节的序列化方式: 0表示JDK,1表示json

        // 总字节数目 = 16(如果不是2的幂可以填充)
        out.writeBytes(magicNum);             // 4字节的协议魔数
        out.writeByte(version);                // 1字节的协议版本
        out.writeByte(serialWay);              // 1字节的序列化方式: 0表示JDK,1表示json
        out.writeByte(msg.getMessageType());   //  1字节指令类型
        out.writeInt(msg.getSequenceId());     // 4字节序列号

        //objectOutputStream:把对象转成字节数据的输出到文件中保存，对象的输出过程称为序列化，可实现对象的持久存储
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ObjectOutputStream oos = new ObjectOutputStream(bos);
        oos.writeObject(msg);
        byte[] content = bos.toByteArray();

        out.writeInt(content.length);              // 写入对象序列化的后的字节数组长度
        out.writeByte(0xff);                       //  填充字符，凑满2的幂为16
        out.writeBytes(content);                   // 写入对象序列化数组

        outList.add(out);
    }

    @Override
    protected void decode(ChannelHandlerContext ctx, ByteBuf in, List<Object> out) throws Exception {
        int magicNum = in.readInt();
        byte version = in.readByte();
        byte serialType = in.readByte();

        byte messageType = in.readByte();
        int sequenceId = in.readInt();

        int length = in.readInt();
        byte padding = in.readByte();
        byte[] arr = new byte[length];
        in.readBytes(arr,0,length);


        ObjectInputStream ois = new ObjectInputStream(new ByteArrayInputStream(arr));
        Message message = (Message)ois.readObject();
        out.add(message);
        log.debug("魔数：{} 版本：{} 序列化方式：{} 消息类型：{} 序列ID：长度：{} {}",magicNum,version,serialType,messageType,sequenceId,length);
        log.debug("{}",message);
    }
}

测试代码中将可复用的handler实例单独提出来：

package application.protocol_design.protocol;

import application.protocol_design.message.LoginRequestMessage;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufAllocator;
import io.netty.channel.embedded.EmbeddedChannel;
import io.netty.handler.codec.LengthFieldBasedFrameDecoder;
import io.netty.handler.logging.LoggingHandler;

import java.util.logging.Handler;

// 测试编码/解码的handler,这里将可以复用的handler实例提取出来
public class TestMessageCodec_version {
    public static void main(String[] args) throws Exception {
        // 能够服用的实例对象单独定义
        LoggingHandler LOGIN_HANDLER = new LoggingHandler();
        MessageCodecSharable PROTOCOL = new MessageCodecSharable();
        EmbeddedChannel ch = new EmbeddedChannel();
        // 打印日志的handler
        ch.pipeline().addLast(LOGIN_HANDLER);
        // 基于长度的handler解决传输过程中的粘包和半包问题
        ch.pipeline().addLast(new LengthFieldBasedFrameDecoder(1024,11,4,1,0));
        // 自定义的协议的编码/解码器handler
        ch.pipeline().addLast(PROTOCOL);

        // 测试出站编码
        System.out.println("\n测试自定义协议的编码\n");


        LoginRequestMessage m = new LoginRequestMessage("god","dog");
        ch.writeOutbound(m);

        System.out.println("\n测试自定义协议的解码\n");
        // 测试入站解码
        ByteBuf bf = ByteBufAllocator.DEFAULT.buffer();
        new MessageCodec().encode(null,m,bf);
        // 这里将一条消息分两次发送，如果没有基于长度字段的编码器LengthFieldBasedFrameDecoder,那么会报错
        ByteBuf b1 = bf.slice(0,100);
        ByteBuf b2 = bf.slice(100,bf.readableBytes()-100); b1.retain();
        ch.writeInbound(b1); ch.writeInbound(b2);
    }
}

参考资料

Netty基础视频教程