前一篇中我们介绍了使用RabbitMQ Java Client访问RabbitMQ的方法。但是使用这种方式访问RabbitMQ,开发者在程序中需要自己管理Connection,Channel对象,Consumer对象的创建,销毁,这样会非常不方便。我们下面介绍使用spring AMQP连接RabbitMQ,进行消息的接收和发送。
Spring AMQP是一个Spring子项目,它提供了访问基于AMQP协议的消息服务器的解决方案。它包含两部分,spring-ampq是基于AMQP协议的消息发送和接收的高层实现,spring-rabbit是基于RabbitMQ的具体实现。这两部分我们下面都会使用到。
Spring-AMQP中的基础类/接口
spring-amqp中定义了几个基础类/接口,Message,Exchange,Queue,Binding
Message
- public class Message implements Serializable
- {
- private final MessageProperties messageProperties;
- private final byte[] body;
spring-amqp中的Message类类似于javax的Message类,封装了消息的Properties和消息体。
Exchange
spring-amqp定义了Exchange接口
- public interface Exchange extends Declarable {
- //Exchange名称
- String getName();
- //Exchange的类型
- String getType();
- //Exchange是否持久化
- boolean isDurable();
- //Exchange不再被使用时(没有任何绑定的情况下),是否由RabbitMQ自动删除
- boolean isAutoDelete();
- //Exchange相关的参数
- Map<String, Object> getArguments();
这个接口和RabbitMQ Client中的Exchange类相似。 spring-amqp中的Exchange继承关系如下图所示
AbstractExchange类是所有Exchange类的父类,实现Exchange接口的具体方法。 CustomExchange针对用户自定义的Exchange对象。其他四个Exchange类,分别对应四种Exchange。 我们在Spring配置文件中配置Exchange对象时,使用的就是这几种Exchange类。
Queue
spring-amqp定义了Queue类,和RabbitMQ Client中的Queue相似,对应RabbitMQ中的消息队列。
- public class Queue extends AbstractDeclarable {
- private final String name;
- private final boolean durable;
- private final boolean exclusive;
- private final boolean autoDelete;
- private final java.util.Map<java.lang.String, java.lang.Object> arguments;
- public Queue(String name, boolean durable, boolean exclusive, boolean autoDelete) {
- this(name, durable, exclusive, autoDelete, null);
- }
Binding
Binding类是对RabbitMQ中Exchange-Exchange以及Exchange-Queue绑定关系的抽象。
- public class Binding extends AbstractDeclarable
- {
- public enum DestinationType {
- QUEUE, EXCHANGE;
- }
- private final String destination;
- private final String exchange;
- private final String routingKey;
- private final Map<String, Object> arguments;
- private final DestinationType destinationType;
- public Binding(String destination, DestinationType destinationType, String exchange, String routingKey,
- Map<String, Object> arguments) {
- this.destination = destination;
- this.destinationType = destinationType;
- this.exchange = exchange;
- this.routingKey = routingKey;
- this.arguments = arguments;
- }
对照RabbitMQ Java Client中Channel接口的queueBind和ExchangeBind方法
- Exchange.BindOk exchangeBind(String destination, String source, String routingKey, Map<String, Object> arguments)
- Queue.BindOk queueBind(String queue, String exchange, String routingKey, Map<String, Object> arguments)
我们可以看出Binding类实际是对底层建立的Exchange-Queue和Exchange-Exchange绑定关系的高层抽象记录类,它使用枚举类型DestinationType区分Exchange-Queue和Exchange-Exchange两种绑定。
Spring AMQP搭建消费者应用
消费者应用程序框架搭建
我们接下来使用spring-amqp搭建一个RabbitMQ的消费者Web应用,我们先创建一个maven webapp应用程序,再添加一个dependency。
- <dependency>
- <groupId>org.springframework.amqp</groupId>
- <artifactId>spring-rabbit</artifactId>
- <version>1.6.5.RELEASE</version>
- </dependency>
spring-rabbit库的引入是为了使用它里面的RabbitAdmin类,创建Exchange,Queue和Binding对象,在导入这个库的时候同时引入了 spring-ampq和rabbitmq-client的库,不需要另行导入。
在src/main/resources目录下创建application.properties文件,用于记录RabbitMQ的配置信息。
- mq.ip=localhost
- mq.port=5672
- mq.userName=rabbitmq_consumer
- mq.password=123456
- mq.virutalHost=test_vhosts
在src/main/resource目录下创建applicationContext.xml文件:
- <?xml version="1.0" encoding="UTF-8"?>
- <beans xmlns="http://www.springframework.org/schema/beans"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xmlns:context="http://www.springframework.org/schema/context"
- xmlns:util="http://www.springframework.org/schema/util"
- xsi:schemaLocation="
- http://www.springframework.org/schema/beans
- http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
- http://www.springframework.org/schema/util
- http://www.springframework.org/schema/util/spring-util-4.0.xsd
- http://www.springframework.org/schema/context
- http://www.springframework.org/schema/context/spring-context-4.0.xsd" >
- <context:annotation-config/>
- <context:property-placeholder
- ignore-unresolvable="true" location="classpath*:/application.properties" />
- <!--从RabbitMQ Java Client创建RabbitMQ连接工厂对象-->
- <bean id="rabbitMQConnectionFactory" class="com.rabbitmq.client.ConnectionFactory">
- <property name="username" value="${mq.userName}" />
- <property name="password" value="${mq.password}" />
- <property name="host" value="${mq.ip}" />
- <property name="port" value="${mq.port}" />
- <property name="virtualHost" value="${mq.virutalHost}" />
- </bean>
- <!--基于RabbitMQ连接工厂对象构建spring-rabbit的连接工厂对象Wrapper-->
- <bean id="connectionFactory" class="org.springframework.amqp.rabbit.connection.CachingConnectionFactory">
- <constructor-arg name="rabbitConnectionFactory" ref="rabbitMQConnectionFactory" />
- </bean>
- <!--构建RabbitAmdin对象,它负责创建Queue/Exchange/Bind对象-->
- <bean id="rabbitAdmin" class="org.springframework.amqp.rabbit.core.RabbitAdmin">
- <constructor-arg name="connectionFactory" ref="connectionFactory" />
- <property name="autoStartup" value="true"></property>
- </bean>
- <!--构建Rabbit Template对象,用于发送RabbitMQ消息,本程序使用它发送返回消息-->
- <bean id="rabbitTemplate" class="org.springframework.amqp.rabbit.core.RabbitTemplate">
- <constructor-arg name="connectionFactory" ref="connectionFactory" />
- </bean>
- <!--RabbitMQ消息转化器,用于将RabbitMQ消息转换为AMQP消息,我们这里使用基本的Message Converter -->
- <bean id="serializerMessageConverter"
- class="org.springframework.amqp.support.converter.SimpleMessageConverter" />
- <!--Message Properties转换器,用于在spring-amqp Message对象中的Message Properties和RabbitMQ的
- Message Properties对象之间互相转换 -->
- <bean id="messagePropertiesConverter"
- class="org.springframework.amqp.rabbit.support.DefaultMessagePropertiesConverter" />
- <!--定义AMQP Queue-->
- <bean id="springMessageQueue" class="org.springframework.amqp.core.Queue">
- <constructor-arg name="name" value="springMessageQueue" />
- <constructor-arg name="autoDelete" value="false" />
- <constructor-arg name="durable" value="true" />
- <constructor-arg name="exclusive" value="false" />
- <!--定义AMQP Queue创建所需的RabbitAdmin对象-->
- <property name="adminsThatShouldDeclare" ref="rabbitAdmin" />
- <!--判断是否需要在连接RabbitMQ后创建Queue-->
- <property name="shouldDeclare" value="true" />
- </bean>
- <!--定义AMQP Exchange-->
- <bean id="springMessageExchange" class="org.springframework.amqp.core.DirectExchange">
- <constructor-arg name="name" value="springMessageExchange" />
- <constructor-arg name="durable" value="true" />
- <constructor-arg name="autoDelete" value="false" />
- <!--定义AMQP Queue创建所需的RabbitAdmin对象-->
- <property name="adminsThatShouldDeclare" ref="rabbitAdmin" />
- <!--判断是否需要在连接RabbitMQ后创建Exchange-->
- <property name="shouldDeclare" value="true" />
- </bean>
- <util:map id="emptyMap" map-class="java.util.HashMap" />
- <!--创建Exchange和Queue之间的Bind-->
- <bean id="springMessageBind" class="org.springframework.amqp.core.Binding">
- <constructor-arg name="destination" value="springMessageQueue" />
- <constructor-arg name="destinationType" value="QUEUE" />
- <constructor-arg name="exchange" value="springMessageExchange" />
- <constructor-arg name="routingKey" value="springMessage" />
- <constructor-arg name="arguments" ref="emptyMap" />
- </bean>
- <!--侦听springMessageQueue队列消息的Message Listener-->
- <bean id="consumerListener"
- class="com.qf.rabbitmq.listener.RabbitMQConsumer" />
- <!--创建侦听springMessageQueue队列的Message Listener Container-->
- <bean id="messageListenerContainer"
- class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
- <property name="messageConverter" ref="serializerMessageConverter" />
- <property name="connectionFactory" ref="connectionFactory" />
- <property name="messageListener" ref="consumerListener" />
- <property name="queues" ref="springMessageQueue" />
- <!--设置消息确认方式为自动确认-->
- <property name="acknowledgeMode" value="AUTO" />
- </bean>
- </beans>
我们定义了侦听消息队列的Message Listener类RabbitMQConsumer
- public class RabbitMQConsumer implements MessageListener
- {
- @Autowired
- private MessagePropertiesConverter messagePropertiesConverter;
- @Override
- public void onMessage(Message message)
- {
- try
- {
- //spring-amqp Message对象中的Message Properties属性
- MessageProperties messageProperties = message.getMessageProperties();
- //使用Message Converter将spring-amqp Message对象中的Message Properties属性
- //转换为RabbitMQ 的Message Properties对象
- AMQP.BasicProperties rabbitMQProperties =
- messagePropertiesConverter.fromMessageProperties(messageProperties, "UTF-8");
- System.out.println("The message's correlationId is:" + rabbitMQProperties.getCorrelationId());
- String messageContent = null;
- messageContent = new String(message.getBody(),"UTF-8");
- System.out.println("The message content is:" + messageContent);
- }
- catch (UnsupportedEncodingException e) {
- e.printStackTrace();
- }
- }
- }
上面的Listener类是实现了MessageListener接口的类,当容器接收到消息后,会自动触发onMessage方法。 如果我们想使用普通的POJO类作为Message Listener,需要引入org.springframework.amqp.rabbit.listener.adapter.MessageListenerAdapter类
- public class MessageListenerAdapter extends AbstractAdaptableMessageListener {
- public MessageListenerAdapter(Object delegate) {
- doSetDelegate(delegate);
- }
- }
这里的delegate对象就是我们的POJO对象。 假设我们定义一个Delegate类ConsumerDelegate
- public class ConsumerDelegate
- {
- public void processMessage(Object message)
- {
- //这里接收的消息对象仅是消息体,不包含MessageProperties
- //如果想获取带MessageProperties的消息对象,需要在Adpater中
- //定义MessageConverter属性。
- String messageContent = message.toString();
- System.out.println(messageContent);
- }
- }
在applicationContext.xml中定义Adapter对象,引用我们的Delegate对象。
- <bean id="consumerDelegate"
- class="com.qf.rabbitmq.listener.ConsumerDelegate" />
- <bean id="consumerListenerAdapter"
- class="org.springframework.amqp.rabbit.listener.adapter.MessageListenerAdapter">
- <property name="delegate" ref="consumerDelegate" />
- <!--指定delegate处理消息的默认方法 -->
- <property name="defaultListenerMethod" value="processMessage" />
- </bean>
最后将Message Listener Container中的Message Listener指向Adapter对象。
- <bean id="messageListenerContainer"
- class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
- <property name="messageConverter" ref="serializerMessageConverter" />
- <property name="connectionFactory" ref="connectionFactory" />
- <!--设置Message Listener为Adapter对象 -->
- <property name="messageListener" ref="consumerListenerAdapter"/>
- <property name="queues" ref="springMessageQueue" />
- <property name="acknowledgeMode" value="AUTO" />
- </bean>
启动Web应用后,我们从启动日志信息可以看出应用连接上了RabbitMQ服务器
从RabbitMQ的管理界面(用rabbitmq_consumer用户登录)可以看到springMessageExchange和springMessageQueue已经创建,绑定关系也已经创建。
Consumer Tag自定义
连接springMessageQueue的消费者Tag是RabbitMQ随机生成的Tag名
如果我们想设置消费者Tag为指定Tag,我们可以在Message Listener Container中 设置自定义consumer tag strategy。首先我们需要定义一个Consumer Tag Strategy类,它实现了ConsumerTagStrategy接口。
- public class CustomConsumerTagStrategy implements ConsumerTagStrategy
- {
- @Override
- public String createConsumerTag(String queue) {
- String consumerName = "Consumer1";
- return consumerName + "_" + queue;
- }
- }
在applicationContext.xml中设定自定义ConsumerTagStrategy
- <bean id="consumerTagStrategy" class="com.qf.rabbitmq.strategy.CustomConsumerTagStrategy" />
- <!--创建侦听springMessageQueue队列的Message Listener Container-->
- <bean id="messageListenerContainer"
- class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
- <property name="messageConverter" ref="serializerMessageConverter" />
- <property name="connectionFactory" ref="connectionFactory" />
- <property name="messageListener" ref="consumerListener" />
- <property name="queues" ref="springMessageQueue" />
- <property name="acknowledgeMode" value="AUTO" />
- <property name="consumerTagStrategy" ref="consumerTagStrategy" />
- </bean>
再次启动Web应用,查看RabbitMQ管理界面,我们可以看到Consumer Tag已经变成“Consumer1_springMessageQueue”,正如我们在CustomConsumerTagStrategy中设定的那样。
消费者应用接收消息验证
- ConnectionFactory factory = new ConnectionFactory();
- factory.setHost("localhost");
- factory.setPort(5672);
- factory.setUsername("rabbitmq_producer");
- factory.setPassword("123456");
- factory.setVirtualHost("test_vhosts");
- //创建与RabbitMQ服务器的TCP连接
- connection = factory.newConnection();
- channel = connection.createChannel();
- String message = "First Web RabbitMQ Message";
- String correlationId = UUID.randomUUID().toString();
- AMQP.BasicProperties props = new AMQP.BasicProperties
- .Builder()
- .correlationId(correlationId)
- .build();
- channel.basicPublish("springMessageExchange","springMessage", props, message.getBytes());
启动消费者Web应用,从控制台输出信息可以看到消费者接收到了生产者发送的消息。
设置消息手动确认模式
到目前为止,消费者端的Web应用对消息的确认是自动确认模式,如果我们想改为手动确认方式,需要做以下两点改动:
1)修改applicationContext.xml文件中Message Listener Container的acknowledgeMode属性的值为MANUAL。
- <bean id="messageListenerContainer"
- class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
- ......
- <property name="acknowledgeMode" value="MANUAL" />
- </bean>
2)将自定义的Message Listener类从实现org.springframework.amqp.core.MessageListener接口,改为实现 org.springframework.amqp.rabbit.core.ChannelAwareMessageListener接口,实现它的 onMessage(Message,Channel)方法。
- public class RabbitMQConsumer implements ChannelAwareMessageListener
- {
- ...........
- @Override
- public void onMessage(Message message, Channel channel)
- {
- try
- {
- //spring-amqp Message对象中的Message Properties属性
- MessageProperties messageProperties = message.getMessageProperties();
- //使用Message Converter将spring-amqp Message对象中的Message Properties属性
- //转换为RabbitMQ 的Message Properties对象
- AMQP.BasicProperties rabbitMQProperties =
- messagePropertiesConverter.fromMessageProperties(messageProperties, "UTF-8");
- System.out.println("The message's correlationId is:" + rabbitMQProperties.getCorrelationId());
- String messageContent = null;
- messageContent = new String(message.getBody(),"UTF-8");
- System.out.println("The message content is:" + messageContent);
- channel.basicAck(messageProperties.getDeliveryTag(), false);
- }
- catch (IOException e) {
- e.printStackTrace();
- }
- }
- }
onMessage方法的最后一句代码调用Channel.basicAck方法对消息进行手动确认。再次运行生产者和消费者程序后,我们登录管理界面,从管理界面中可以看到springMessageQueue队列中未确认消息条数 (图中Unacked列)为0条,说明消费者接收消息后已经手动确认。
RPC模式设置
如果生产者和消费者Web应用之间使用RPC模式,即消费者接收消息后要向指定Exchange/Queue发送返回消息,我们需要修改生产者和消费者的程序。 消费者程序修改点如下:
1)在applicationContext.xml中定义返回消息对应的Exchange,Queue和Bind。
- <!--定义AMQP Reply Queue-->
- <bean id="springReplyMessageQueue" class="org.springframework.amqp.core.Queue">
- <constructor-arg name="name" value="springReplyMessageQueue" />
- <constructor-arg name="autoDelete" value="false" />
- <constructor-arg name="durable" value="true" />
- <constructor-arg name="exclusive" value="false" />
- <property name="adminsThatShouldDeclare" ref="rabbitAdmin" />
- <property name="shouldDeclare" value="true" />
- </bean>
- <!--定义AMQP Reply Exchange-->
- <bean id="springReplyMessageExchange" class="org.springframework.amqp.core.DirectExchange">
- <constructor-arg name="name" value="springReplyMessageExchange" />
- <constructor-arg name="durable" value="true" />
- <constructor-arg name="autoDelete" value="false" />
- <!--定义AMQP Queue创建所需的RabbitAdmin对象-->
- <property name="adminsThatShouldDeclare" ref="rabbitAdmin" />
- <property name="shouldDeclare" value="true" />
- </bean>
- <!--创建Reply Exchange和Reply Queue之间的Bind-->
- <bean id="springReplyMessageBind" class="org.springframework.amqp.core.Binding">
- <constructor-arg name="destination" value="springReplyMessageQueue" />
- <constructor-arg name="destinationType" value="QUEUE" />
- <constructor-arg name="exchange" value="springReplyMessageExchange" />
- <constructor-arg name="routingKey" value="springReplyMessage" />
- <constructor-arg name="arguments" ref="emptyMap" />
- </bean>
- public void onMessage(Message message, Channel channel) {
- try
- {
- ......................
- String replyMessageContent = "Consumer1 have received the message '" + messageContent + "'";
- channel.basicPublish(rabbitMQProperties.getReplyTo(), "springReplyMessage",
- rabbitMQProperties, replyMessageContent.getBytes());
- ......................
这里发送返回消息直接使用接收消息时创建的Channel通道,不过如果我们的Message Listener类是继承自MessageListener接口,无法获得Channel对象时,我们需要使用RabbitTemplate对象进行返回消息的发送(我们前面已经在applicationContext.xml中定义了这个对象)
- public class RabbitMQConsumer implements MessageListener
- {
- @Autowired
- private MessagePropertiesConverter messagePropertiesConverter;
- @Autowired
- private RabbitTemplate rabbitTemplate;
- @Override
- public void onMessage(Message message)
- {
- ..........
- //创建返回消息的RabbitMQ Message Properties
- AMQP.BasicProperties replyRabbitMQProps =
- new AMQP.BasicProperties("text/plain",
- "UTF-8",
- null,
- 2,
- 0, rabbitMQProperties.getCorrelationId(), null, null,
- null, null, null, null,
- null, null);
- //创建返回消息的信封头
- Envelope replyEnvelope =
- new Envelope(messageProperties.getDeliveryTag(), true,
- "springReplyMessageExchange", "springReplyMessage");
- //创建返回消息的spring-amqp Message Properties属性
- MessageProperties replyMessageProperties =
- messagePropertiesConverter.toMessageProperties(replyRabbitMQProps,
- replyEnvelope,"UTF-8");
- //构建返回消息(spring-amqp消息)
- Message replyMessage = MessageBuilder.withBody(replyMessageContent.getBytes())
- .andProperties(replyMessageProperties)
- .build();
- rabbitTemplate.send("springReplyMessageExchange","springReplyMessage", replyMessage);
- String correlationId = UUID.randomUUID().toString();
- AMQP.BasicProperties props = new AMQP.BasicProperties
- .Builder()
- .correlationId(correlationId)
- .replyTo("springReplyMessageExchange")
- .build();
- channel.basicPublish("springMessageExchange","springMessage", props, message.getBytes());
- QueueingConsumer replyCustomer = new QueueingConsumer(channel);
- channel.basicConsume("springReplyMessageQueue",true,"Producer Reply Consumer", replyCustomer);
- String responseMessage = null;
- while(true)
- {
- QueueingConsumer.Delivery delivery = replyCustomer.nextDelivery();
- String messageCorrelationId = delivery.getProperties().getCorrelationId();
- if (messageCorrelationId != null && messageCorrelationId.equals(correlationId))
- {
- responseMessage = new String(delivery.getBody());
- System.out.println("The reply message's correlation id is:" + messageCorrelationId);
- break;
- }
- }
- if(responseMessage != null)
- {
- System.out.println("The repsonse message is:'" + responseMessage + "'");
- }
消费者控制台
生产者控制台
消费者并发数设置
到目前为止,消费者Web应用消费消息时,只有一个消费者接收并消费springMessageQueue队列的消息(如下图所示)如果发送的消息量比较大时,我们需要增加消费者的数目。
增加消费者数目要修改Message Listener Container的concurrentConsumers和maxConcurrentConsumers属性,concurrentConsumers属性是Message Listener Container创建时创建的消费者数目,maxConcurrentConsumers属性是容器最大的消费者数目,我们下面把这两个属性都设置为5,使Message Listener Container中有5个消费者,同时修改CustomerConsumerTagStrategy类,在Tag中加入线程名,以区分不同的消费者。
- <bean id="messageListenerContainer"
- class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
- ............
- <property name="consumerTagStrategy" ref="consumerTagStrategy" />
- <property name="concurrentConsumers" value="5" />
- <property name="maxConcurrentConsumers" value="5" />
- </bean>
- public class CustomConsumerTagStrategy implements ConsumerTagStrategy
- {
- @Override
- public String createConsumerTag(String queue) {
- String consumerName = "Consumer_" + Thread.currentThread().getName();
- return consumerName + "_" + queue;
- }
- }
修改生产者程序,循环发送50条消息
- ReplyConsumer replyCustomer = new ReplyConsumer(channel);
- channel.basicConsume("springReplyMessageQueue",true,"Producer Reply Consumer", replyCustomer);
- for(int i=0; i<50; i++)
- {
- String correlationId = UUID.randomUUID().toString();
- String message = "Web RabbitMQ Message " + i;
- AMQP.BasicProperties props =
- new AMQP.BasicProperties
- .Builder()
- .contentType("text/plain")
- .deliveryMode(2)
- .correlationId(correlationId)
- .replyTo("springReplyMessageExchange")
- .build();
- channel.basicPublish("springMessageExchange","springMessage", props, message.getBytes());
- }
- public class ReplyConsumer extends DefaultConsumer
- {
- public ReplyConsumer(Channel channel)
- {
- super(channel);
- }
- @Override
- public void handleDelivery(String consumerTag,
- Envelope envelope,
- AMQP.BasicProperties properties,
- byte[] body)
- throws IOException
- {
- String consumerName = properties.getAppId();
- String replyMessageContent = new String(body, "UTF-8");
- System.out.println("The reply message's sender is:" + consumerName);
- System.out.println("The reply message is '" + replyMessageContent + "'");
- }
- }
- public void onMessage(Message message, Channel channel)
- {
- try
- {
- String consumerTag = messageProperties.getConsumerTag();
- String replyMessageContent = consumerTag + " have received the message '" + messageContent + "'";
- AMQP.BasicProperties replyRabbitMQProps =
- new AMQP.BasicProperties("text/plain",
- "UTF-8",
- null,
- 2,
- 0, rabbitMQProperties.getCorrelationId(), null, null,
- null, null, null, null,
- consumerTag, null);
- .............
- public class CustomConsumerTagStrategy implements ConsumerTagStrategy
- {
- @Override
- public String createConsumerTag(String queue) {
- String consumerName = "Consumer_" + Thread.currentThread().getName();
- return consumerName;
- }
- }
消费者消息预取数设置
上述的消费者Web应用中,每个消费者每次从队列中获取1条消息,如果我们想让每个消费者一次性从消息队列获取多条消息,需要修改Message Listener Container的prefetchCount属性,这样可以提高RabbitMQ的消息处理吞吐量。- <span style="font-size:10px;"><bean id="messageListenerContainer"
- class="org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer">
- <property name="prefetchCount" value="5" />
- </bean></span>
- /**
- * Request a specific prefetchCount "quality of service" settings
- * for this channel.
- *
- * @see #basicQos(int, int, boolean)
- * @param prefetchCount maximum number of messages that the server
- * will deliver, 0 if unlimited
- * @throws java.io.IOException if an error is encountered
- */
- void basicQos(int prefetchCount) throws IOException
这个方法设置从Channel上一次性可以读取多少条消息,我们在Container设置的PrefetchCount值为5,表示从一个消费者Channel上,一次性可以与预读取5条消息,按我们上面设置的5个消费者,5个消费者Channel计算,一次性可以预读取25条消息。为了证实这一点,我们修改消费者的代码,延长它处理一条消息的时间。
需要说明的是,对于每个消费者而言,只有一条预取的消息被接收且确认后,消费者才会再从消息队列中读取消息,并不是消费者在消息没有确认完成前,每次都从队列里预读取prefetchCount条消息。
- public void onMessage(Message message, Channel channel) {
- try
- {
- ...........
- String messageContent = null;
- messageContent = new String(message.getBody(),"UTF-8");
- String consumerTag = messageProperties.getConsumerTag();
- String replyMessageContent = consumerTag + " have received the message '" + messageContent + "'";
- Thread.sleep(60000);
- ...........
- rabbitTemplate.send("springReplyMessageExchange","springReplyMessage", replyMessage);
- channel.basicAck(messageProperties.getDeliveryTag(), false);
我们在onMessage方法中添加Thread.sleep(60000),使得处理一条消息时间时间大于1分钟,便于查看消息预取的效果,而且使用手动确认方式。
生产者程序改为一次性发送200条消息。
启动生产者程序,发送200条消息,我们可以看到springMessageQueue队列里有200条处于Ready状态的消息
启动消费者程序,我们可以看到springMessageQueue队列里有25条消息被预取了,Ready状态的消息从200条变成了175条,而未确认状态的消息数(Unacked列)变成了25条,即25条被预取,但是没有被确认的消息。
过了一段时间,等5个消费者确认了5条消息后,又从消息队列预读取了5条消息,Ready状态的消息数变成170条,这时的消息队列的消息数如下图所示:
未确认的消息数仍然是25条,但是总的消息数变成了195条,表示已经有5条消息被处理且确认了。
随着消息逐渐被处理,确认,消费者会逐渐从消息队列预取新的消息,直到所有的消息都被处理和确认完成。
rabbit标签使用
上面的消费者Web应用使用了Spring传统的beans元素定义,spring-rabbit提供了rabbit namespace,我们可以在applicationContext.xml中使用rabbit:xxx形式的元素标签,简化我们的xml配置。 我们首先在applicationContext.xml的namespace定义中添加rabbit namespace定义:- <beans xmlns="http://www.springframework.org/schema/beans"
- xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
- xmlns:context="http://www.springframework.org/schema/context"
- xmlns:util="http://www.springframework.org/schema/util"
- xmlns:rabbit="http://www.springframework.org/schema/rabbit"
- xsi:schemaLocation="
- http://www.springframework.org/schema/beans
- http://www.springframework.org/schema/beans/spring-beans-4.0.xsd
- http://www.springframework.org/schema/util
- http://www.springframework.org/schema/util/spring-util-4.0.xsd
- http://www.springframework.org/schema/rabbit
- http://www.springframework.org/schema/rabbit/spring-rabbit-1.6.xsd
- http://www.springframework.org/schema/context
- http://www.springframework.org/schema/context/spring-context-4.0.xsd" >
- <pre name="code" class="html"><span style="font-size:10px;"><rabbit:connection-factory id ="connectionFactory" connection-factory="rabbitMQConnectionFactory" /></span></pre>
- <pre></pre>
- <span style="color:rgb(51,51,51); font-family:Arial,sans-serif"><span style="font-size:10px">修改RabbitAdmin bean对象定义,使用rabbit:admin标签</span></span>
- <pre></pre>
- <pre></pre>
- <pre></pre>
- <rabbit:admin id="rabbitAdmin" connection-factory="connectionFactory" auto-startup="true"/>
- <rabbit:template connection-factory="connectionFactory" />
MessageConverter和MessageProperties对象没有对应的rabbit标签,仍然使用bean标签。
修改Queue,Exchange和Bind定义,分别使用rabbit:queue,rabbit:exchange标签,Bind的内容放到了Exchange bean定义内部。
- <rabbit:queue id="springMessageQueue" name="springMessageQueue" auto-delete="false"
- durable="true" exclusive="false" auto-declare="false" declared-by="rabbitAdmin" />
- <rabbit:direct-exchange id="springMessageExchange" name="springMessageExchange" durable="true"
- auto-declare="false" auto-delete="false" declared-by="rabbitAdmin">
- <rabbit:bindings>
- <rabbit:binding queue="springMessageQueue" key="springMessage"></rabbit:binding>
- </rabbit:bindings>
- </rabbit:direct-exchange>
- <rabbit:listener-container message-converter="serializerMessageConverter"
- connection-factory="connectionFactory"
- acknowledge="manual"
- consumer-tag-strategy="consumerTagStrategy"
- concurrency="5"
- max-concurrency="5"
- prefetch="5">
- <rabbit:listener ref="consumerListener" queues="springMessageQueue"/>
- </rabbit:listener-container>
- <rabbit:listener ref="consumerListener" queue-names="springMessageQueue"/>
这里如果Listener关联多个队列,设置queues属性或者queue-names属性时可以用逗号进行分割,例如:
- <pre name="code" class="html" style="color: rgb(51, 51, 51);"><rabbit:listener ref="consumerListener" queue-names="messageQueue1,messageQueue2"/></pre>
- <pre></pre>
- <pre></pre>
- <pre></pre>
- <pre></pre>
使用rabbit标签虽然可以简化RabbitMQ相关对象的bean定义,但是它也有局限性:
1)标签对应的bean对象类型是固定的,例如rabbit:listener-container标签对应的Listener Container是SimpleMessageListenerContainer类,如果我们想使用其他MessageListenerContainer类或者自定义Message Listener Container类,就不能使用rabbit标签。
2)有的标签无法设置id和name属性,这样一旦有多个同类型的bean对象定义时,就不能使用rabbit标签。
RabbitMQ的Channel和Connection缓存
- <rabbit:connection-factory id ="connectionFactory"
- connection-factory="rabbitMQConnectionFactory"
- cache-mode="CHANNEL"
- channel-cache-size="30" />
- <rabbit:connection-factory id ="connectionFactory"
- connection-factory="rabbitMQConnectionFactory"
- cache-mode="CONNECTION"
- connection-cache-size="10" />
- public class CachingConnectionFactory extends AbstractConnectionFactory
- {
- ................
- private volatile int connectionLimit = Integer.MAX_VALUE;
这个属性默认值是Integer.MAX_VALUE,可以理解为无上限,我们可以在applicationContext.xml中设置这个值为10。
- <rabbit:connection-factory id ="connectionFactory"
- connection-factory="rabbitMQConnectionFactory"
- connection-limit="10"
- cache-mode="CONNECTION"
- connection-cache-size="10" />
- <span style="font-size:10px;"><rabbit:listener-container
- .............
- concurrency="4"
- max-concurrency="4">
- <rabbit:listener ref="Listener1" queues="messageQueue1"/>
- <rabbit:listener ref="Listener2" queues="messageQueue2"/>
- <rabbit:listener ref="Listener3" queues="messageQueue3"/>
- </rabbit:listener-container></span>
例如上面的Container中,一共定义了三个Listener,每个Listener的并发数是4,总的并发数为12,超过了上线10,因此抛出以下异常:
- 一月 03, 2017 10:15:28 上午 org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer redeclareElementsIfNecessary
- 严重: Failed to check/redeclare auto-delete queue(s).
- org.springframework.amqp.AmqpTimeoutException: Timed out attempting to get a connection
- at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:575)
- ..............
Spring AMQP的重连机制
我们在使用1中介绍了RabbitMQ Java Client提供的重连机制,Spring AMQP也提供了重连机制。我们可以使用Rabbit Java Client的重连设置,我们修改applicationContext.xml中“rabbitMQConnectionFactory”的重连属性设置。- <bean id="rabbitMQConnectionFactory" class="com.rabbitmq.client.ConnectionFactory">
- ...................
- <property name="automaticRecoveryEnabled" value="true" />
- <property name="topologyRecoveryEnabled" value="true" />
- <property name="networkRecoveryInterval" value="60000" />
- </bean>
我们启动消费者应用程序,打开管理页面,可以看到消费者应用创建了5个Connection,每个Connection下分别创建了一个Channel,对应5个Consumer。
我们停止RabbitMQ服务器,可以看到消费者控制台输出连接异常信息,不停试图恢复Consumer。
重新启动RabbitMQ服务器,从日志信息可以看出连接被重置,消费者被恢复。
点开一条Channel进去,可以看到连接Channel的Consumer Tag与最初的Consumer Tag也不一致,这可能是因为我们使用了自定义ConsumerTagStrategy,使用线程名为Tag名的原因。
我们也可以禁用RabbitMQ Java Client的重连设置,设置automaticRecoveryEnabled和topologyRecoveryEnabled属性为false。
- <span style="font-size:10px;"><bean id="rabbitMQConnectionFactory" class="com.rabbitmq.client.ConnectionFactory">
- <property name="automaticRecoveryEnabled" value="false" />
- <property name="topologyRecoveryEnabled" value="false" />
- </bean></span>
当我们重启RabbitMQ服务器后,发现只有4个Connection恢复,5个Channel被恢复,但是有两个Channel复用同一个Connection,这一点与 使用RabbitMQ Java Client的重连机制时有所不同。
当执行RabbitMQ重连时,Message Listener Container也会对Consumer进行重新恢复,它的恢复间隔是由recoveryBackOff属性决定的。
- public class SimpleMessageListenerContainer extends AbstractMessageListenerContainer
- implements ApplicationEventPublisherAware {
- ..........
- private BackOff recoveryBackOff = new FixedBackOff(DEFAULT_RECOVERY_INTERVAL, FixedBackOff.UNLIMITED_ATTEMPTS);
SimpleMessageListenerContainer类的recoveryBackOff属性对象有两个属性,一个是恢复间隔,默认值是DEFAULT_RECOVERY_INTERVAL常量(5000ms,即每5秒试图进行一次恢复),还有一个尝试恢复次数,默认值是FixedBackOff.UNLIMITED_ATTEMPTS(Long.MaxValue,可以认为是无限次尝试)。我们可以根据需要 设置自己的recoveryBackOff属性,例如下面我们把恢复间隔设置为60000ms,尝试次数设置为100次。
- <bean id="backOff" class="org.springframework.util.backoff.FixedBackOff">
- <constructor-arg name="interval" value="60000" />
- <constructor-arg name="maxAttempts" value="100" />
- </bean>
- <rabbit:listener-container message-converter="serializerMessageConverter"
- ..........
- recovery-back-off="backOff">
- <rabbit:listener ref="consumerListener" queues="springMessageQueue"/>
- </rabbit:listener-container>
修改后启动消费者应用,停掉RabbitMQ服务器,我们从异常日志可以看出Message Listener Container的重试间隔变成了1分钟,而不是默认的5000ms。(为了便于查看重试间隔起见,我们将Container的并发数调整为1)