• RabbitMQ延时任务


    概念:

    消息的TTL(Time To Live)
    消息的TTL就是消息的存活时间。RabbitMQ可以对队列和消息分别设置TTL。对队列设置就是队列没有消费者连着的保留时间,也可以对每一个单独的消息做单独的设置。超过了这个时间,我们认为这个消息就死了,称之为死信。
    如果队列设置了,消息也设置了,那么会取小的。所以一个消息如果被路由到不同的队列中,这个消息死亡的时间有可能不一样(不同的队列设置)。这里单讲单个消息的TTL,因为它才是实现延迟任务的关键。
    可以通过设置消息的expiration字段或者x-message-ttl属性来设置时间,两者是一样的效果。
    消息扔到队列中后,过了设置的限定时间,如果没有被消费,它就死了。不会被消费者消费到。这个消息后面的,没有“死掉”的消息对顶上来,被消费者消费。
    死信在队列中并不会被删除和释放,它会被统计到队列的消息数中去。单靠死信还不能实现延迟任务,还要靠Dead Letter Exchange。

    Dead Letter Exchanges
    Exchage的概念在这里就不在赘述,可以从这里进行了解。一个消息在满足如下条件下,会进死信路由,记住这里是路由而不是队列,一个路由可以对应很多队列。
    1. 一个消息被Consumer拒收了,并且reject方法的参数里requeue是false。也就是说不会被再次放在队列里,被其他消费者使用。
    2. 上面的消息的TTL到了,消息过期了。
    3. 队列的长度限制满了。排在前面的消息会被丢弃或者扔到死信路由上。
    Dead Letter Exchange其实就是一种普通的exchange,和创建其他exchange没有两样。只是在某一个设置Dead Letter Exchange的队列中有消息过期了,会自动触发消息的转发,发送到Dead Letter Exchange中去。

    实现延迟队列:
    延迟任务通过消息的TTL和Dead Letter Exchange来实现。我们需要建立2个队列,一个用于发送消息,一个用于消息过期后的转发目标队列。

    发布者Code:

            public void PublishDelayMessage<T>(T message, int expireMinutes, bool durable = true) where T : class
            {
                if (expireMinutes <= 0)
                {
                    throw new ArgumentException("expireMinutes 必须大于0");
                }
    
                using (var channel = connection.CreateModel())
                {
                    var body = Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(message));
    
                    //创建默认的死信交换机
                    string receiveExchangeName = $"{typeof(T).FullName}.DelayReceive";
                    string receiveQueueName = $"{receiveExchangeName}.{expireMinutes}";
                    channel.ExchangeDeclare(exchange: receiveExchangeName, type: "direct", durable: durable);
    
                    string bufferExchange = $"{typeof(T).FullName}.DelayBuffer";
                    string bufferQueueName = $"{bufferExchange}.{expireMinutes}";
                    channel.ExchangeDeclare(exchange: bufferExchange, type: "direct", durable: durable);
    
                    //创建消息缓冲队列,在这个队列里面实现消息的过期转发
                    var properties = channel.CreateBasicProperties();
                    //properties.Expiration = (expireMinutes * 60000).ToString();
                    properties.Expiration= (expireMinutes * 6000).ToString();
                    Dictionary<string, object> arguments = new Dictionary<string, object>();
                    arguments.Add("x-dead-letter-exchange", receiveExchangeName);
                    arguments.Add("x-dead-letter-routing-key", receiveQueueName);
                    channel.QueueDeclare(queue: bufferQueueName, durable: durable, exclusive: false, autoDelete: false, arguments: arguments);
                    channel.QueueBind(queue: bufferQueueName, exchange: bufferExchange, routingKey: bufferQueueName);
    
                    //这个队列用于消息在缓冲队列中过期后转发的目标队列
                    channel.QueueDeclare(queue: receiveQueueName, durable: durable, exclusive: false, autoDelete: false, arguments: null);
                    channel.QueueBind(queue: receiveQueueName, exchange: receiveExchangeName, routingKey: receiveQueueName);
    
                    channel.BasicPublish(exchange: bufferExchange, routingKey: bufferQueueName, basicProperties: properties, body: body);
                }
            }

    消费订阅:

      /// <summary>
            /// 订阅延迟消息
            /// </summary>
            /// <typeparam name="T"></typeparam>
            /// <param name="onMessage"></param>
            /// <param name="expireMinutes"></param>
            /// <param name="durable"></param>
            /// <returns></returns>
            public bool SubscribeDelayMessage<T>(Action<T> onMessage, int expireMinutes, bool durable = true) where T : class
            {
                if (expireMinutes <= 0)
                {
                    throw new ArgumentException("expireMinutes 必须大于0");
                }
    
                var channel = connection.CreateModel();
    
                string receiveExchangeName = $"{typeof(T).FullName}.DelayReceive";
                string receiveQueueName = $"{receiveExchangeName}.{expireMinutes}";
                channel.ExchangeDeclare(exchange: receiveExchangeName, type: "direct", durable: durable);
    
                channel.QueueDeclare(queue: receiveQueueName, durable: durable, exclusive: false, autoDelete: false, arguments: null);
                channel.QueueBind(queue: receiveQueueName, exchange: receiveExchangeName, routingKey: receiveQueueName);
    
                var consumer = new EventingBasicConsumer(channel);
                consumer.Received += (model, ea) =>
                {
                    try
                    {
                        var message = Encoding.UTF8.GetString(ea.Body);
                        onMessage(JsonConvert.DeserializeObject<T>(message));
                    }
                    catch
                    {
    
                    }
                };
                channel.BasicConsume(queue: receiveQueueName, noAck: true, consumer: consumer);
                return true;
            }

    虽然没贴出全部的代码,但是最核心的已经有了

    1,设置消息的过期时间

    2.设置缓冲队列,并且在消息过期以后转发到真实的路由中

    看Wireshark抓包分析:

    1.过期时间

    可以看到发布消息的properties里面设置了expiration

    2.过期转发

    可以看到缓冲队列在声明的时候,设置了arguments

    里面配置了x-dead-letter-exchange,x-dead-letter-routing-key

    这样缓冲队列里面的消息过期以后,就将消息转发给配置的对应配置的交换机路由。

    优化,上面的版本中,延时的时间不同,就必须要创建新的死信交换机,死信消息队列,接受交换机,接受消息队列,这样大大的浪费了资源

    现在对其进行优化:

     /// <summary>
            /// 发布延迟消息,当createSpecifyReceiveQueue=true时,延迟订阅参数expireMinutes必须与本方法的expireMinutes保持一致
            /// </summary>
            /// <typeparam name="T"></typeparam>
            /// <param name="message"></param>
            /// <param name="expireMinutes"></param>
            /// <param name="createSpecifyReceiveQueue"></param>
            /// <param name="durable"></param>
            public void PublishDelayMessage<T>(T message, int expireMinutes, bool createSpecifyReceiveQueue = false, bool durable = true) where T : class
            {
                if (expireMinutes <= 0)
                {
                    throw new ArgumentException("expireMinutes 必须大于0");
                }
    
                using (var channel = connection.CreateModel())
                {
                    var body = Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(message));
    
                    //创建默认的死信交换机
                    string receiveExchangeName = $"{typeof(T).FullName}.DelayReceive";
                    string receiveQueueName = "";
                    //创建指定的接收队列
                    if (createSpecifyReceiveQueue)
                    {
                        receiveQueueName = $"{receiveExchangeName}.{expireMinutes}";
                    }
                    else
                    {
                        receiveQueueName = $"{receiveExchangeName}.DelayDefaultReceive";
                    }
                    channel.ExchangeDeclare(exchange: receiveExchangeName, type: "direct", durable: durable);
    
                    string bufferExchange = $"{typeof(T).FullName}.DelayBuffer";
                    string bufferQueueName = "";
                    if (createSpecifyReceiveQueue)
                    {
                        bufferQueueName = $"{bufferExchange}.{expireMinutes}";
                    }
                    else
                    {
                        bufferQueueName = $"{bufferExchange}.Default.{expireMinutes}";
                    }
                    channel.ExchangeDeclare(exchange: bufferExchange, type: "direct", durable: durable);
    
                    //创建消息缓冲队列,在这个队列里面实现消息的过期转发
                    var properties = channel.CreateBasicProperties();
                    properties.Expiration = (expireMinutes * 60000).ToString();
                    Dictionary<string, object> arguments = new Dictionary<string, object>();
                    arguments.Add("x-dead-letter-exchange", receiveExchangeName);
                    arguments.Add("x-dead-letter-routing-key", receiveQueueName);
                    channel.QueueDeclare(queue: bufferQueueName, durable: durable, exclusive: false, autoDelete: false, arguments: arguments);
                    channel.QueueBind(queue: bufferQueueName, exchange: bufferExchange, routingKey: bufferQueueName);
    
                    //这个队列用于消息在缓冲队列中过期后转发的目标队列
                    channel.QueueDeclare(queue: receiveQueueName, durable: durable, exclusive: false, autoDelete: false, arguments: null);
                    channel.QueueBind(queue: receiveQueueName, exchange: receiveExchangeName, routingKey: receiveQueueName);
    
                    channel.BasicPublish(exchange: bufferExchange, routingKey: bufferQueueName, basicProperties: properties, body: body);
                }
            }
    
            /// <summary>
            /// 订阅延迟消息,注意:如果expireMinutes>0,发布的延迟消息的参数createSpecifyReceiveQueue必须设置为true
            /// </summary>
            /// <typeparam name="T"></typeparam>
            /// <param name="onMessage"></param>
            /// <param name="expireMinutes"></param>
            /// <param name="durable"></param>
            /// <returns></returns>
            public bool SubscribeDelayMessage<T>(Action<T> onMessage, int expireMinutes = 0, bool durable = true) where T : class
            {
                var channel = connection.CreateModel();
    
                string receiveExchangeName = $"{typeof(T).FullName}.DelayReceive";
                string receiveQueueName = "";
                channel.ExchangeDeclare(exchange: receiveExchangeName, type: "direct", durable: durable);
                if (expireMinutes <= 0)
                {
                    receiveQueueName = $"{receiveExchangeName}.DelayDefaultReceive";
                    channel.QueueDeclare(queue: receiveQueueName, durable: durable, exclusive: false, autoDelete: false, arguments: null);
                }
                else
                {
                    receiveQueueName = $"{receiveExchangeName}.{expireMinutes}";
                    channel.QueueDeclare(queue: receiveQueueName, durable: durable, exclusive: false, autoDelete: false, arguments: null);
                }
                channel.QueueBind(queue: receiveQueueName, exchange: receiveExchangeName, routingKey: receiveQueueName);
    
                var consumer = new EventingBasicConsumer(channel);
                consumer.Received += (model, ea) =>
                {
                    try
                    {
                        var message = Encoding.UTF8.GetString(ea.Body);
                        onMessage(JsonConvert.DeserializeObject<T>(message));
                    }
                    catch
                    {
    
                    }
                };
                channel.BasicConsume(queue: receiveQueueName, noAck: true, consumer: consumer);
                return true;
            }

  • 相关阅读:
    Python实现TCP服务端的并发
    python程序中的线程操作
    jmeter命令行执行脚本_动态参数设置
    App客户端性能测试点总结
    App功能测试点总结
    jmeter中生成UUID作为唯一标识符
    Python Unittest进行接口测试的简单示例
    jmeter接口测试中的用例数据分离
    博客园看板娘的简单添加
    (转)后端服务性能压测实践
  • 原文地址:https://www.cnblogs.com/hongdada/p/7410850.html
Copyright © 2020-2023  润新知