• netty 的流量整形写测试


     如何测试netty 的流量整形呢? 因为 其实服务端、客户端都可以限速,读写也可以限速,有多个组合,所以比较复杂;
     
    考虑一种最简单的情况,就是单独对客户端写进行限速,如下:
     
    服务端等待客户端连接;接受客户端请求之后,打印每秒收到的数据有多少, 然后简单打印一下速率;怎么做呢? 服务端对每一个客户端的channel保存一个计数器用于统计收到的总流量大小consumeMsgLength,然后启动一个定时器,每秒执行一次,任务是把consumeMsgLength 打印一下,然后置为0 (相当于是重置);
     
    客户端连接到服务端,然后发送请求,比如是每秒随机写10M,也就是 10M/s 的写(当然也可以 设置一个范围,如 1~10 M/s ,客户端固定速度的写是为了 测试方便起见, 实际情况可能非常不同),但是限速是每秒1M,也就是 rate = 1M/s;那么 客户端理论上发送的数据 会有多少, 应该是1M/s, 多余的数据到哪里去了? 其实控制好就不会有这种问题。因为客户端应该进行控制,一定要根据 客制化的可写性判断,允许写才写!
     
    下面是我测试通过的代码以及说明。
     
    服务端关键代码
        ServerBootstrap sbs = new ServerBootstrap();
                // 配置nio服务参数
                sbs.group(bossGroup, workerGroup)
                   .channel(NioServerSocketChannel.class) // 说明一个新的Channel如何接收进来的连接
                   .option(ChannelOption.SO_BACKLOG, 128) // tcp最大缓存链接个数
                   .childOption(ChannelOption.SO_KEEPALIVE, true) //保持连接
                   .handler(new LoggingHandler(LogLevel.INFO)) // 打印日志级别
                   .childHandler(new ChannelInitializer<SocketChannel>() {
                        @Override
                        protected void initChannel(SocketChannel socketChannel) throws Exception {
                            // 处理接收到的请求
                            ChannelPipeline pipeline = socketChannel.pipeline();
                            // 基于换行符号
    //                        pipeline.addLast(new LineBasedFrameDecoder(1024)); // 这个会导致 channelRead0 没有执行,因为 数据一直在接收之中.. 等待一个换行
                            // 解码转String,注意调整自己的编码格式GBK、UTF-8
                            pipeline.addLast(new StringDecoder(Charset.forName("GBK")));
                            // 解码转String,注意调整自己的编码格式GBK、UTF-8
                            pipeline.addLast(new StringEncoder(Charset.forName("GBK")));
    
                            //流量整形
    //                        pipeline.addLast(new ChannelTrafficShapingHandler(10, 10));
                            pipeline.addLast(new MyServerCommonHandler( ));

    管道处理器:

    import io.netty.channel.*;
    
    import java.util.Date;
    import java.util.Optional;
    import java.util.concurrent.atomic.AtomicLong;
    import java.util.function.Consumer;
    
    public class MyServerCommonHandler extends ChannelInboundHandlerAdapter {
    
        protected final int KB = 1024;//  * 1024;
        protected String tempStr;
        protected AtomicLong consumeMsgLength;
        protected Runnable counterTask;
        protected boolean sentFlag;
    
        @Override
        public void channelWritabilityChanged(ChannelHandlerContext ctx) throws Exception {
            if(ctx.channel().isWritable() && !sentFlag) {
                System.out.println(" ###### 重新开始写数据 ######");
            } else {
                System.out.println(" ===== 写暂停 =====");
            }
        }
        
        @Override
        public void handlerAdded(ChannelHandlerContext ctx) throws Exception {
            System.out.println("MyServerCommonHandler.handlerAdded");
            consumeMsgLength = new AtomicLong();
            counterTask = () -> {
              while (true) {
                  try {
                      Thread.sleep(1000);
                  } catch (InterruptedException e) {
    
                  }
    
                  long length = consumeMsgLength.getAndSet(0);
                  System.out.println("*** " + ctx.channel().remoteAddress() + " rate(KB/S):" + (length) + "  " + new Date());
              }
            };
            StringBuilder builder = new StringBuilder();
            for (int i = 0; i < KB; i++) {
                builder.append("abcdefghijklmnopqrstuvwxyz");
            }
            tempStr = builder.toString();
            super.handlerAdded(ctx);
        }
    
        @Override
        public void channelActive(ChannelHandlerContext ctx) throws Exception {
            new Thread(counterTask).start();
        }
    
    
        protected abstract void sentData(ChannelHandlerContext ctx);
    
        @Override
        public void channelRead(ChannelHandlerContext ctx, Object msg) throws Exception {
            if (msg instanceof String) {
                int length = ((String) msg).length();
                // // 为什么这里 是默认 15s 一次, 因为io.netty.handler.traffic.AbstractTrafficShapingHandler.DEFAULT_MAX_TIME = 15 000
                long l = consumeMsgLength.addAndGet(length);// 为什么每次都是 10240 ?  因为客户端写过来的数据就是这么多
                System.out.println("===== receive client " + " " + " size:" + l);
            }
    //        System.out.println("===== receive client msg : " + msg + " 
      " + l);
            super.channelRead(ctx, msg);
        }
    }
    客户端
       @Override
                  protected void initChannel(SocketChannel socketChannel) throws Exception {                  
                        // 处理来自服务端的响应信息
                      ChannelPipeline pipeline = socketChannel.pipeline();
                      ChannelTrafficShapingHandler channelTrafficShapingHandler = new MyChannelTrafficShapingHandler((long) (0.001 * KB), 1 * KB);
                      // 解码转String,注意调整自己的编码格式GBK、UTF-8
                      pipeline.addLast(new StringDecoder(Charset.forName("GBK")));
                      // 解码转String,注意调整自己的编码格式GBK、UTF-8
                      pipeline.addLast(new StringEncoder(Charset.forName("GBK")));
                      pipeline
                        .addLast("channelTrafficShapingHandler",channelTrafficShapingHandler)
                        .addLast(new ClientOutHandler())

    上面设置是:writeLimit = 0.001 * KB,也就是 每秒最多写 0.001 * KB,

    管道处理器:

    import com.zx.sms.testNetty.Student;
    import io.netty.buffer.ByteBuf;
    import io.netty.channel.ChannelHandlerContext;
    import io.netty.channel.ChannelInboundHandlerAdapter;
    import io.netty.channel.ChannelOutboundHandlerAdapter;
    import io.netty.channel.ChannelPromise;
    import io.netty.util.ReferenceCountUtil;
    
    import java.net.SocketAddress;
    import java.nio.charset.Charset;
    import java.util.Date;
    
    /**
     * 读取服务器返回的响应信息
     * @author luo
     *
     */
    public class ClientOutHandler extends ChannelOutboundHandlerAdapter {
    
        @Override
        public void write(ChannelHandlerContext ctx, Object msg, ChannelPromise promise) throws Exception {
            System.out.println("write = " + new Date() + " ctx.channel().isWritable()  " + ctx.channel().isWritable());
            super.write(ctx, msg, promise);
        }
    
        @Override
        public void disconnect(ChannelHandlerContext ctx, ChannelPromise promise) throws Exception {
            System.out.println("ClientOutHandler.disconnect" + new Date());
            super.disconnect(ctx, promise);
        }
    
        @Override
        public void read(ChannelHandlerContext ctx) throws Exception {
            System.out.println("ClientOutHandler.read" + new Date());
            super.read(ctx);
        }
    
        @Override
        public void close(ChannelHandlerContext ctx, ChannelPromise promise) throws Exception {
            System.out.println("ClientOutHandler.close" + new Date());
            super.close(ctx, promise);
        }
    
        @Override
        public void connect(ChannelHandlerContext ctx, SocketAddress remoteAddress, SocketAddress localAddress, ChannelPromise promise) throws Exception {
            System.out.println("ctx = [" + ctx + "], remoteAddress = [" + remoteAddress + "], localAddress = [" + localAddress + "], promise = [" + promise + "]" + new Date());
            super.connect(ctx, remoteAddress, localAddress, promise);
    
            new Thread() {
                @Override
                public void run() {
                    StringBuilder builder = new StringBuilder();
                    for (int i = 0; i < 1024*1; i++) {
                        builder.append("abcdefghij");
                    }
                    int cnt = 0;
                    String o = builder.toString();
                    while (cnt < 1000) {
                        try {
                            Thread.sleep(100);
                        } catch (InterruptedException e) {
                            e.printStackTrace();
                        }
                        boolean writable = ctx.channel().isWritable();
                        /*
                            因为io.netty.handler.traffic.AbstractTrafficShapingHandler.DEFAULT_MAX_TIME
                            所以 无论, 都会写一次。
                         */
                        if (writable) {
                            ctx.channel().writeAndFlush(o);
                        } else{
    //                        System.out.println("ClientOutHandler.run ctx.channel().isWritable() no");
                        }
    
                        cnt++;
                    }
                }
            }.start();
        }
    
        @Override
        public void flush(ChannelHandlerContext ctx) throws Exception {
            super.flush(ctx);
        }
    
        @Override
        public void deregister(ChannelHandlerContext ctx, ChannelPromise promise) throws Exception {
            System.out.println("ctx = [" + ctx + "], promise = [" + promise + "]    " + new Date() + " ctx.channel().isWritable()  " + ctx.channel().isWritable());
            super.deregister(ctx, promise);
        }
    
    
        @Override
        public void handlerAdded(ChannelHandlerContext ctx) throws Exception {
            System.out.println("ctx = [" + ctx + "] " + new Date());
            super.handlerAdded(ctx);
        }
    }
     
    和guava 的RateLimiter 不同,netty的 TrafficShapingHandler也是 先等待(最久等待默认是15s ),然后 时间到了,然后读或者写, 不管数据量有多大。
     
    也就是说, 如果 rate = 1M/s, 第一次写100M数据,按理说是需要 100s, 但是实际上只会等待15s, 然后就把100 m, 一次性写过去。 也就是说, 存在突刺的情况!
     
    当然, 其实现原理也是差别很大。
     
    测试,服务端日志:
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:21 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:22 CST 2021
    ===== receive client   size:10240
    MyServerCommonHandler.channelReadComplete     Sat May 29 20:38:22 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):10240  Sat May 29 20:38:23 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:24 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:25 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:26 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:27 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:28 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:29 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:30 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:31 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:32 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:33 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:34 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:35 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:36 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:37 CST 2021
    ===== receive client   size:10240
    MyServerCommonHandler.channelReadComplete     Sat May 29 20:38:37 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):10240  Sat May 29 20:38:38 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:39 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:40 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:41 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:42 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:43 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:44 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:45 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:46 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:47 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:48 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:49 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:50 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:51 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:52 CST 2021
    ===== receive client   size:10240
    MyServerCommonHandler.channelReadComplete     Sat May 29 20:38:52 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):10240  Sat May 29 20:38:53 CST 2021
    *** /127.0.0.1:33533 rate(KB/S):0  Sat May 29 20:38:54 CST 2021
     
    客户端打印:
    write = Sat May 29 20:37:37 CST 2021 ctx.channel().isWritable()  true
    write = Sat May 29 20:37:52 CST 2021 ctx.channel().isWritable()  true
    write = Sat May 29 20:38:07 CST 2021 ctx.channel().isWritable()  true
    write = Sat May 29 20:38:22 CST 2021 ctx.channel().isWritable()  true
    write = Sat May 29 20:38:37 CST 2021 ctx.channel().isWritable()  true
    write = Sat May 29 20:38:52 CST 2021 ctx.channel().isWritable()  true
    write = Sat May 29 20:39:07 CST 2021 ctx.channel().isWritable()  true
    可见, 每秒统计速度,都是0,但是每15秒就写一次。 receive client size:10240 打印的就是 实际接受的数据, 符合预期!
     
     
    需要注意的是, 客户端写限速之后,客户端执行ctx.channel().writeAndFlush(o) 方法需要先判断一下 是否可写。否则限速就完全无效!!!
     
    因为 ChannelTrafficShapingHandler 实际上不做限制,而是 仅仅给对应的通道做一个标记!
     
    如果改一下, 把
    ClientOutHandler#connect 的 builder.append("abcdefghij"); 改为 builder.append("a");
    new MyChannelTrafficShapingHandler((long) (0.001 * KB), 1 * KB); 改为 new MyChannelTrafficShapingHandler((long) (1 * KB), 1 * KB);
     
    也就是 每次写 1024 B,每秒10次即每秒写 10240 B即 10 KB, 限速也是 1 KB, 那么, 应该来说,是每秒 可以写 10次,观察 客户端日志:
    write =  Sat May 29 23:14:34 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:34 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:34 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:39 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:39 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:39 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:39 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:39 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:44 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:44 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:44 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:44 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:44 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:49 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:49 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:49 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:49 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:49 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:54 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:54 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:54 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:54 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:54 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:59 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:59 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:59 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:59 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:14:59 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:04 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:04 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:04 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:04 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:04 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:09 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:09 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:09 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:09 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:09 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:14 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:14 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:14 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:14 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:14 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:19 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:19 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:19 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:19 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:19 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:24 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:15:24 CST 2021 ctx.channel().isWritable()  true
    服务端日志:
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:25 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:25 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:26 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:26 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:27 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:27 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:28 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:28 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:29 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:29 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:30 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:30 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:31 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:31 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:32 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:32 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:33 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:33 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:34 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:34 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:35 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:35 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:36 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:36 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:37 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:37 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:38 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:38 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:39 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:39 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:45:40 CST 2021
    *** /127.0.0.1:40279 rate(KB/S):1024  Sat May 29 23:45:40 CST 2021

     可见并不符合预期!

    发现并不是 预设的那样, 实际隔5秒 写一次,一次写5行(5个write操作)! 但是 平均来说 它确实是每秒 只能写一次!
     
    竟然有假的 可写性修改。 为什么会这样, 从 io.netty.handler.traffic.AbstractTrafficShapingHandler#checkWriteSuspend 找到答案:
    void checkWriteSuspend(ChannelHandlerContext ctx, long delay, long queueSize) {
        if (queueSize > maxWriteSize || delay > maxWriteDelay) {
            setUserDefinedWritability(ctx, false);
        }
    }
    maxWriteSize默认值是4M,maxWriteDelay 默认值是4s, 就是说默认只有 当前写大小超过了4M或总延迟超过4s,才会实际的修改 管道可写性, 否则不会改变。 虽然少量的延迟不会立即修改可写性,所以写操作会理解提交到管道之后,所以看到上面的一秒5行的写日志, 而且服务端能够立即收到数据( 每秒一次一KB)。但是效果延迟会积累,所以,超过5s之后的写 还是需要等到 5s 之后,所以看到上面的 每隔5行的写新的日志 。
     
     
    其他不变,如果 改为 new MyChannelTrafficShapingHandler((long) (2 * KB), 1 * KB); 那么 服务端每秒收到2次写,每次1KB, 即 rate = 2k/s ;
    服务端:
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:53:23 CST 2021
    ===== receive client   size:2048
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:53:24 CST 2021
    *** /127.0.0.1:41167 rate(KB/S):2048  Sat May 29 23:53:24 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:53:24 CST 2021
    ===== receive client   size:2048
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:53:25 CST 2021
    *** /127.0.0.1:41167 rate(KB/S):2048  Sat May 29 23:53:25 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:53:25 CST 2021
    ===== receive client   size:2048
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:53:26 CST 2021
    *** /127.0.0.1:41167 rate(KB/S):2048  Sat May 29 23:53:26 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:53:26 CST 2021
    ===== receive client   size:2048
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:53:27 CST 2021
    *** /127.0.0.1:41167 rate(KB/S):2048  Sat May 29 23:53:27 CST 2021
    ===== receive client   size:1024
    MyServerCommonHandler.channelReadComplete     Sat May 29 23:53:27 CST 2021
    ===== receive client   size:2048

    发现其中  ===== receive client 打印了 2次, 分别是1024 2048 ,rate(KB/S)才打印一次 固定是 2048,这个是因为 服务端收到2次(每次手动1KB数据)后才会清零计数器。 rate 是正确的,其他打印也是正确的!

    客户端:
    write =  Sat May 29 23:51:52 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:52 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:52 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:52 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:52 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:52 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:52 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:53 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:53 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:53 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:53 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:57 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:57 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:57 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:57 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:57 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:57 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:58 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:58 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:58 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:51:58 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:02 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:02 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:02 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:02 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:02 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:02 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:03 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:03 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:03 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:03 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:07 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:07 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:07 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:07 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:07 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:07 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:08 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:08 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:08 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:08 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:15 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:16 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:16 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:16 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:16 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:16 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:16 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:16 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:16 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:16 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:16 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:21 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:21 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:21 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:21 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:21 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:21 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:22 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:22 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:22 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:22 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:26 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:26 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:26 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:26 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:26 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:26 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:27 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:27 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:27 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:27 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:34 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:35 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:35 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:35 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:35 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:35 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:35 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:35 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:35 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:35 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:35 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:40 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:40 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:40 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:40 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:40 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:40 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:41 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:41 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:41 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:41 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:45 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:45 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:45 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:45 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:45 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:45 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:46 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:46 CST 2021 ctx.channel().isWritable()  true
    write =  Sat May 29 23:52:46 CST 2021 ctx.channel().isWritable()  true

     观察发现 基本上 平均是 每5秒写10次,一次一行一kB! 就是翻倍了, 即 avg rate = 2k/s!

    如果 客户端一次写的 内容 超过4M 它也不会被发开为 多次写, 而是在等待之后立即写。 不管写数据量是多少,默认超过15s 就要写完!
     
    总体来说, 符合预期,限流确实是有效果了, 但是 限流结果并不是 平滑直线,而是可能出现突刺 !
     
    ---------------------------------------------  更正  -----------------------------------------
     
    更正, 仔细测试,观察上面服务端的日志发现,服务端收到是数据是平滑的, 是没有突刺的! 仅仅从上面客户端的日志来看,似乎是“每5秒写10次,一次一行一kB”,似乎存在突刺, 但是 为什么服务端收到的数据不是的呢?
    其实不是这样的! 再次检查源码, 注意@1 处,发现:
        void submitWrite(final ChannelHandlerContext ctx, final Object msg,
                final long size, final long delay, final long now,
                final ChannelPromise promise) {
            final ToSend newToSend;
            // write order control
            synchronized (this) {
                if (delay == 0 && messagesQueue.isEmpty()) {
                    trafficCounter.bytesRealWriteFlowControl(size);
                    ctx.write(msg, promise);
                    return;
                }
                newToSend = new ToSend(delay + now, msg, promise);
                messagesQueue.addLast(newToSend);
                queueSize += size;
                checkWriteSuspend(ctx, delay, queueSize); // @1
            }
            final long futureNow = newToSend.relativeTimeAction;
            ctx.executor().schedule(new Runnable() {
                @Override
                public void run() {
                    sendAllValid(ctx, futureNow);
                }
            }, delay, TimeUnit.MILLISECONDS);
        }

    checkWriteSuspend 方法其实没有返回值,就是说它 仅仅是一个优化,即使没有立即的设置管道的可写性,也仅仅是影响可写性,如果此时有数据继续可过来, 此时会写到本地内存缓存中去! 而且,只要delay != 0 ,那么就不会立即return,那么就会提交一个延迟任务(不管是否可写),任务的执行是时间是未来时间:  futureNow 。

    如果把写速度不变,限速提高到1MB,那么就是相当于没有了限速。

    按理说,当前提交了1个延迟任务,sendAllValid 方法就会执行对应的1次的写! 但是从sendAllValid源码来看,它有一个循环, 当内存中的缓冲队列messagesQueue 写完为空之后,又会release WriteSuspended ( 后面的需要等待前面的写完) 如果第1秒的时候 一个请求req1 过来,写很多数据,需要延迟10s 即第11s执行; 如果第2秒的时候 又一个请求req2 过来,写很多数据,需要延迟3s 即第5s执行,那么 req2 是会比req1 先执行吗? 还是说必须排队到 req1之后? 观察 io.netty.handler.traffic.TrafficCounter#writeTimeToWait(long, long, long, long) 可知,返回的wait 即 delay 是在本窗口中递增的数值,是第二个情况, 所以 req2  实际是 10 +3 第13s执行, 所以 第10秒 写完 req1之后, messagesQueue 还有一个数据,不为空 ,所以 保证了 release WriteSuspended 不是很频繁, 不会反复的被无谓的执行。 

    所以一般就是先wait的先write,但是 并发的情况下就不一定了! 所以上面 还有synchronized  加锁。

    综上,基本上来说 客户端的数据 也都是按照设定的时间写出去的,大部分是平滑的。 但是如果发送的速度过快或数据量过大超出了限速,那么仍然有可能在超时的时刻有突刺。

     
    futureNow 


    版权声明
    本文原创发表于 博客园,作者为 阿K .     本文欢迎转载,但未经作者同意必须保留此段声明,且在文章页面明显位置给出原文连接,否则视为侵权。
    欢迎关注本人微信公众号:觉醒的码农,或者扫码进群:

  • 相关阅读:
    无主之地1
    A+B Problem(V)
    取石子(七)
    又见拦截导弹
    【Chat】实验 -- 实现 C/C++下TCP, 服务器/客户端 "多人聊天室"
    【Echo】实验 -- 实现 C/C++下UDP, 服务器/客户端 通讯
    【Echo】实验 -- 实现 C/C++下TCP, 服务器/客户端 通讯
    对决
    重建二叉树
    花儿朵朵
  • 原文地址:https://www.cnblogs.com/FlyAway2013/p/14827243.html
Copyright © 2020-2023  润新知