RocketMq producer 发送一条消息所经过的流程

 前言：

　　RocketMq producer 在发送一条消息时候，从 producer --nameSrv -- Broker  中间经过了什么样子的数据交互

开始：

如下是 Producer 发送消息的一个demo例子：

　　　　//1. 初始化 mq producer
        DefaultMQProducer mqProducer =new DefaultMQProducer("iscys-test");
        //2.设置nameServer 地址
        mqProducer.setNamesrvAddr("localhost:9876");
        //3. 开启mq producer,这一步是必须的，会做一些连接初始化检测工作
        mqProducer.start();
        //4.创建 Message
        Message msg = new Message("test-topis", "iscys-test".getBytes());
        //5.发送消息，设置回调，消息发送成功会回调函数
        mqProducer.send(msg, new SendCallback() {

            @Override
            public void onSuccess(SendResult sendResult) {
                //在消息发送成功之后，我们收到broker的响应通知后，会进行回调
                System.out.println("send success");
            }

            @Override
            public void onException(Throwable e) {
                System.out.println("send fail");

            }
        });

构建发送消息：

 public void send(Message msg, SendCallback sendCallback, long timeout)
        throws MQClientException, RemotingException, InterruptedException {
        try {
            //默认异步发送，超时3s
            this.sendDefaultImpl(msg, CommunicationMode.ASYNC, sendCallback, timeout);
        } catch (MQBrokerException e) {
            throw new MQClientException("unknownn exception", e);
        }
    }

从NameSrv 中获取topic 配置的相关信息，比如 broker 地址，队列数 之类的。

 private SendResult sendDefaultImpl(
        Message msg,
        final CommunicationMode communicationMode,
        final SendCallback sendCallback,
        final long timeout
    ) throws MQClientException, RemotingException, MQBrokerException, InterruptedException {
        this.makeSureStateOK();
        Validators.checkMessage(msg, this.defaultMQProducer);

        final long invokeID = random.nextLong();
        long beginTimestampFirst = System.currentTimeMillis();
        long beginTimestampPrev = beginTimestampFirst;
        long endTimestamp = beginTimestampFirst;
        //1.尝试取获取从NameSrv 中获取topic 相关信息
        TopicPublishInfo topicPublishInfo = this.tryToFindTopicPublishInfo(msg.getTopic());
        if (topicPublishInfo != null && topicPublishInfo.ok()) {
            MessageQueue mq = null;
            Exception exception = null;
            SendResult sendResult = null;
            int timesTotal = communicationMode == CommunicationMode.SYNC ? 1 + this.defaultMQProducer.getRetryTimesWhenSendFailed() : 1;
            int times = 0;
            String[] brokersSent = new String[timesTotal];
            for (; times < timesTotal; times++) {
                String lastBrokerName = null == mq ? null : mq.getBrokerName();
                //2.选择一个消息队列，默认为4个，在创建新的Topic时候
                MessageQueue mqSelected = this.selectOneMessageQueue(topicPublishInfo, lastBrokerName);
                if (mqSelected != null) {
                    mq = mqSelected;
                    brokersSent[times] = mq.getBrokerName();
                    try {
                        beginTimestampPrev = System.currentTimeMillis();
                        //3.发送消息
                        sendResult = this.sendKernelImpl(msg, mq, communicationMode, sendCallback, topicPublishInfo, timeout);
                        endTimestamp = System.currentTimeMillis();
                        this.updateFaultItem(mq.getBrokerName(), endTimestamp - beginTimestampPrev, false);
                        switch (communicationMode) {
                            case ASYNC:
                                return null;
                            case ONEWAY:

主要看一下如上代码第一步 尝试获取Topic 信息  tryToFindTopicPublishInfo:

　　　　

　　　1. 会先从 topicPublishInfoTable 缓存中获取topic 配置信息

　　　2.缓存没有，就从NameSrv 中拉取。

　　　3.如果获取到了，则返回。

　　　4.NameSrv 没有得到相关到topic 信息，说明是新到topic ，则就请求获取TBW102 topic 配置信息，这个肯定能获取到，封装使用TBW102的配置。

    private TopicPublishInfo tryToFindTopicPublishInfo(final String topic) {
        //1.从 topicPublishInfoTable 从尝试从Map中获取，如果没有获取到，请求NameSrv
        TopicPublishInfo topicPublishInfo = this.topicPublishInfoTable.get(topic);
        if (null == topicPublishInfo || !topicPublishInfo.ok()) {
            this.topicPublishInfoTable.putIfAbsent(topic, new TopicPublishInfo());
            //2.从NameSrv 中拉取topic 信息
            this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic);
            topicPublishInfo = this.topicPublishInfoTable.get(topic);
        }
       
        if (topicPublishInfo.isHaveTopicRouterInfo() || topicPublishInfo.ok()) {
            //3.说明获取到TOPIC 的信息
            return topicPublishInfo;
        } else {
            //4.如果第2步执行后 NameSrv 中没有topic 信息，获取默认的TBW102 topic 的信息，这个是肯定能获取到的
            this.mQClientFactory.updateTopicRouteInfoFromNameServer(topic, true, this.defaultMQProducer);
            topicPublishInfo = this.topicPublishInfoTable.get(topic);
            return topicPublishInfo;
        }
    }

请求NameSrv 非默认的topic

 public boolean updateTopicRouteInfoFromNameServer(final String topic) {
        //1.非默认的topic ，默认Topic 为TBW102
        return updateTopicRouteInfoFromNameServer(topic, false, null);
    }

执行从NameSrv 获取topic 请求:

　　　　1. 从NameSrv 中获取到 TBW102 的topic 信息，这个一般都是有的。

　　　　2. 新的topic 会从NameSrv 中获取信息，如果不存在，返回false。

public boolean updateTopicRouteInfoFromNameServer(final String topic, boolean isDefault,
        DefaultMQProducer defaultMQProducer) {
        try {
            if (this.lockNamesrv.tryLock(LOCK_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS)) {
                try {
                    TopicRouteData topicRouteData;
                    if (isDefault && defaultMQProducer != null) {
                        // 1.如果请求的是默认的Topic 请求会走到这里
                        topicRouteData = this.mQClientAPIImpl.getDefaultTopicRouteInfoFromNameServer(defaultMQProducer.getCreateTopicKey(),
                            1000 * 3);
                        if (topicRouteData != null) {
                            for (QueueData data : topicRouteData.getQueueDatas()) {
                                int queueNums = Math.min(defaultMQProducer.getDefaultTopicQueueNums(), data.getReadQueueNums());
                                data.setReadQueueNums(queueNums);
                                data.setWriteQueueNums(queueNums);
                            }
                        }
                    } else {
                        // 2.新的Topic 会先从NameSrv 中获取一遍，如果NameSrv 中没有获取到，会抛出异常
                        topicRouteData = this.mQClientAPIImpl.getTopicRouteInfoFromNameServer(topic, 1000 * 3);
                    } 

 从NameSrv 获取到 TopicRouteData 进行如下表的缓存

　　　　　　　　　　brokerAddressTable  --- 从TopicRouteData 得到broker 信息，进行brokerAddressTable 存储

　　　　　　　　　　TopicPublishInfoTable --TopicRouteData 转换为topicPushInfo 存储到 topicPublishInfoTable表中

　　　　　　　　　　topicRouteTable　 ---存储namesrv 得到的 TopicRouteData 　值　

 if (topicRouteData != null) {
                        //1.与缓存做比较，查看topic 相关信息是否有改变
                        TopicRouteData old = this.topicRouteTable.get(topic);
                        boolean changed = topicRouteDataIsChange(old, topicRouteData);
                        if (!changed) {
                            changed = this.isNeedUpdateTopicRouteInfo(topic);
                        } else {
                            log.info("the topic[{}] route info changed, old[{}] ,new[{}]", topic, old, topicRouteData);
                        }

                        if (changed) {
                            TopicRouteData cloneTopicRouteData = topicRouteData.cloneTopicRouteData();
                            //2.broker 信息进行brokerAddrTable 表存储
                            for (BrokerData bd : topicRouteData.getBrokerDatas()) {
                                //缓存
                                this.brokerAddrTable.put(bd.getBrokerName(), bd.getBrokerAddrs());
                            }

                            // Update Pub info
                            {
                                //3.将topicRouteData 转换为TopicPublishInfo ，用于producer
                                TopicPublishInfo publishInfo = topicRouteData2TopicPublishInfo(topic, topicRouteData);
                                //设置为true 标记这个publishInfo是可用的
                                publishInfo.setHaveTopicRouterInfo(true);
                                Iterator<Entry<String, MQProducerInner>> it = this.producerTable.entrySet().iterator();
                                while (it.hasNext()) {
                                    Entry<String, MQProducerInner> entry = it.next();
                                    MQProducerInner impl = entry.getValue();
                                    if (impl != null) {
                                        //更新producer  topicPublishInfoTable表信息
                                        impl.updateTopicPublishInfo(topic, publishInfo);
                                    }
                                }
                            }
                                //4.consumerTable 维护用户Consumer
                            // Update sub info
                            {
                                Set<MessageQueue> subscribeInfo = topicRouteData2TopicSubscribeInfo(topic, topicRouteData);
                                Iterator<Entry<String, MQConsumerInner>> it = this.consumerTable.entrySet().iterator();
                                while (it.hasNext()) {
                                    Entry<String, MQConsumerInner> entry = it.next();
                                    MQConsumerInner impl = entry.getValue();
                                    if (impl != null) {
                                        impl.updateTopicSubscribeInfo(topic, subscribeInfo);
                                    }
                                }
                            }
                            log.info("topicRouteTable.put. Topic = {}, TopicRouteData[{}]", topic, cloneTopicRouteData);
                            //5.topicRouteTable 表存储原始信息，用于定时任务心跳等 
                            this.topicRouteTable.put(topic, cloneTopicRouteData);
                            return true;

获取到topic信息后封装成 TopicPublishInfo：

public class TopicPublishInfo {
    private boolean orderTopic = false;
    //用来检测Topic 在Broker 真实存在的，不存在false
    private boolean haveTopicRouterInfo = false;
    //消息队列的
    private List<MessageQueue> messageQueueList = new ArrayList<MessageQueue>();
    private volatile ThreadLocalIndex sendWhichQueue = new ThreadLocalIndex();
    //请求NameSrv 返回的TOPIC 具体信息 
    private TopicRouteData topicRouteData;

 之后就是消息的发送。

再看producer 启动：

　　说完消息发送，我们再看producer 的启动初始化：

　　　　　1.检查配置信息

　　　　　2.初始化MQClientInstance

　　　　　3.将producer 信息注册到MQClientInstance维护的producerTable 表中

　　　　　4. topicPublishInfoTable 放入默认的topic

　　　　　5.启动MQClientInstance

　　　　　6.producer 状态修改为运行状态

　　　　　7.发送心跳向broker

public void start(final boolean startFactory) throws MQClientException {
        switch (this.serviceState) {
            case CREATE_JUST:
                this.serviceState = ServiceState.START_FAILED;
                //1.检查配置
                this.checkConfig();

                if (!this.defaultMQProducer.getProducerGroup().equals(MixAll.CLIENT_INNER_PRODUCER_GROUP)) {
                    this.defaultMQProducer.changeInstanceNameToPID();
                }
                //2.初始化 MQClientInstance producer 交互的重要信息组件
                this.mQClientFactory = MQClientManager.getInstance().getAndCreateMQClientInstance(this.defaultMQProducer, rpcHook);
                // 3.producerGroup 注册到 producerTable 表中 k= producerName v = this
                boolean registerOK = mQClientFactory.registerProducer(this.defaultMQProducer.getProducerGroup(), this);
                if (!registerOK) {
                    this.serviceState = ServiceState.CREATE_JUST;
                    throw new MQClientException("The producer group[" + this.defaultMQProducer.getProducerGroup()
                        + "] has been created before, specify another name please." + FAQUrl.suggestTodo(FAQUrl.GROUP_NAME_DUPLICATE_URL),
                        null);
                }
                //4.topicPublishInfoTable 表中默认放入TBW102
                this.topicPublishInfoTable.put(this.defaultMQProducer.getCreateTopicKey(), new TopicPublishInfo());

                if (startFactory) {
                    //5.启动工厂
                    mQClientFactory.start();
                }

                log.info("the producer [{}] start OK. sendMessageWithVIPChannel={}", this.defaultMQProducer.getProducerGroup(),
                    this.defaultMQProducer.isSendMessageWithVIPChannel());
                //6.状态改变为启动状态
                this.serviceState = ServiceState.RUNNING;
                break;
            case RUNNING:
            case START_FAILED:
            case SHUTDOWN_ALREADY:
                throw new MQClientException("The producer service state not OK, maybe started once, "
                    + this.serviceState
                    + FAQUrl.suggestTodo(FAQUrl.CLIENT_SERVICE_NOT_OK),
                    null);
            default:
                break;
        }
        //7.发送心跳检测向所有的broker
        this.mQClientFactory.sendHeartbeatToAllBrokerWithLock();
    }

我们主要看  MQClientInstance 创建以及启动：

public class MQClientInstance {
    private final static long LOCK_TIMEOUT_MILLIS = 3000;
    private final Logger log = ClientLogger.getLog();
    //配置信息
    private final ClientConfig clientConfig;
    //创建索引
    private final int instanceIndex;
    //pid
    private final String clientId;
    private final long bootTimestamp = System.currentTimeMillis();
    //producer 维护的group 表，producer 会注册这张表，如上代码启动的第3步
    private final ConcurrentMap<String/* group */, MQProducerInner> producerTable = new ConcurrentHashMap<String, MQProducerInner>();
    //consumer 维护的group 表
    private final ConcurrentMap<String/* group */, MQConsumerInner> consumerTable = new ConcurrentHashMap<String, MQConsumerInner>();
    //admin 维护的group 表
    private final ConcurrentMap<String/* group */, MQAdminExtInner> adminExtTable = new ConcurrentHashMap<String, MQAdminExtInner>();
    //Netty 配置类
    private final NettyClientConfig nettyClientConfig;
    //Netty 客户端
    private final MQClientAPIImpl mQClientAPIImpl;
    //admin
    private final MQAdminImpl mQAdminImpl;
    //从NameSrv 拉取到Topic 相关信息的原始值放在这个表里面
    private final ConcurrentMap<String/* Topic */, TopicRouteData> topicRouteTable = new ConcurrentHashMap<String, TopicRouteData>();
    private final Lock lockNamesrv = new ReentrantLock();
    private final Lock lockHeartbeat = new ReentrantLock();
    //broker 地址表
    private final ConcurrentMap<String/* Broker Name */, HashMap<Long/* brokerId */, String/* address */>> brokerAddrTable =
        new ConcurrentHashMap<String, HashMap<Long, String>>();
    //broker version 表
    private final ConcurrentMap<String/* Broker Name */, HashMap<String/* address */, Integer>> brokerVersionTable =
        new ConcurrentHashMap<String, HashMap<String, Integer>>();
    private final ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor(new ThreadFactory() {
        @Override
        public Thread newThread(Runnable r) {
            return new Thread(r, "MQClientFactoryScheduledThread");
        }
    });
    //请求处理器
    private final ClientRemotingProcessor clientRemotingProcessor;
    //拉去消息服务
    private final PullMessageService pullMessageService;
    //负载均衡服务
    private final RebalanceService rebalanceService;
    //默认Group 的producer 服务
    private final DefaultMQProducer defaultMQProducer;
    //consumer 状态管理
    private final ConsumerStatsManager consumerStatsManager;
    private final AtomicLong sendHeartbeatTimesTotal = new AtomicLong(0);
    //MqClientInstance 启动状态
    private ServiceState serviceState = ServiceState.CREATE_JUST;
    private DatagramSocket datagramSocket;
    private Random random = new Random();

启动MQClientInstance:

  public void start() throws MQClientException {

        synchronized (this) {
            switch (this.serviceState) {
                case CREATE_JUST:
                    this.serviceState = ServiceState.START_FAILED;
                    // 如果没有配置nameSrv 默认从java 配置的propertues 参数路由地址获取信息，阿里就是这么使用的，nameSrv 的获取就相当于一个web 服务
                    if (null == this.clientConfig.getNamesrvAddr()) {
                        this.mQClientAPIImpl.fetchNameServerAddr();
                    }
                    // Start request-response channel
                    //初始化Netty 客户端
                    this.mQClientAPIImpl.start();
                    // Start various schedule tasks
                    //开启一些定时任务
                    this.startScheduledTask();
                    // Start pull service
                    this.pullMessageService.start();
                    // Start rebalance service
                    this.rebalanceService.start();
                    // Start push service  默认group的 producer 启动
                    this.defaultMQProducer.getDefaultMQProducerImpl().start(false);
                    log.info("the client factory [{}] start OK", this.clientId);
                    this.serviceState = ServiceState.RUNNING;
                    break;
                case RUNNING:
                    break;
                case SHUTDOWN_ALREADY:
                    break;
                case START_FAILED:
                    throw new MQClientException("The Factory object[" + this.getClientId() + "] has been created before, and failed.", null);
                default:
                    break;
            }
        }
    }