开源消息中间件DotNetMQ

由于这个开源项目对我这种中间件菜鸟很有帮助，因此，我将官方的说明文档翻译如下：

Introduction

　　In this article, I will introduce a new and independent Open Source Message Queue system that is entirely built in C# and .NET framework 3.5. DotNetMQ is a message broker that has several features including guaranteed delivering, routing, load balancing, server graphs... so on. I will start by explaining messaging concepts and the need for message brokers. Then I will examine what DotNetMQ is and how to use it.

　　在这篇文章，我将介绍一个新的独立的完全基于C#和.Net Framework 3.5 框架开发的开源消息队列系统。DotNetMQ是一个具有包括传送保证、路由、负载均衡、服务器图等等多个特点的消息代理。我将从解析消息概念和消息代理的必要性开始介绍，然后，我将检验DotNetMQ是什么和怎么使用它。

What Is Messaging?

　　Messaging is a way of asynchronous communication of applications running on same or different machines with reliable delivery. Programs communicate by sending packets of data called messages to each other [1].

　　消息传送是一种运行在相同或者不同机器上的带有可靠传输的异步通信程序。程序通过发送称为消息的数据包给其他程序来通信。

　　A message may be a string, a byte array, an object... etc. Typically, a sender (producer) program creates a message and pushes it to a message queue and a receiver (consumer) program gets the message from the queue and processes it. The sender and receiver programs don’t have to be running at the same time, since messaging is an asynchronous process. This is called loosely coupled communication.

　　一个消息可以是一个字符串、一个字节数组、一个对象等等。典型地，一个消息发送者（生产者）程序创造一个信息并把它插到消息队列中去，消息接收者（消费者）程序从消息队列中把消息取出来并使用消息。发送和接收程序必须同时运行，从而，消息传统是一个异步过程。这称为松耦合的通信。

　　On the other hand, a Web Service method call (Remote Method Invocation) is a type of tightly coupled and synchronous communication (both applications have to be running and available during the whole communication; if the Web Service is offline or an error occurs during the method call, the client application gets an exception).

　　另一方面，一个Web服务方法的调用是一种紧耦合与同步的通信（发送端、接收端程序与Web服务必须在整个通信过程中同时运行并可用，如果Web服务处于离线状态或者在函数调用期间发生了一个错误，客户端程序会得到一个异常）。

　　In the figure above, two applications communicate over a message queue in a loosely coupled manner. If the receiver consumes messages slower than the sender produces it, the message count on the queue will increase. Also, the receiver may be offline while the sender is sending messages. In this situation, the receiver gets the messages from the queue when it becomes online (when it starts and joins the queue).

　　Message Queues are typically provided by Message Brokers. A Message Broker（消息代理） is a standalone application (service) that other applications connect to and send/receive messages. A Message Broker is responsible to store messages until a receiver receives them. A Message Broker can route messages（消息路由） across machines to deliver a message to the destination application and can try delivering the message until the receiver correctly handles it. A Message Broker is sometimes called a Message Oriented Middleware (MOM面向消息的中间件) or simply Message Queue (MQ).

What is DotNetMQ?

DotNetMQ is an open source Message Broker that has several features:

• Persistent or non-persistent messaging. 持久性或非持久性消息传递。
• Guaranteed delivery of persistent messages even in a system crash. 即使在系统崩溃时持久信息的保证交付。
• Automatic and manual routing of messages in a custom machine graph.在一个自定义的机器图表的自动或手动的消息路由。
• Supports multiple databases (MS SQL Server, MySQL, SQLite, and memory-based storage for now).支持多种数据库（MS SQL Server, MySql, Sqlite和其他目前基于内存存储的数据库）。
• Supports don’t store, direct send style messaging. 支持存储发送或者直接发送两种消息传送。
• Supports Request/Reply style messaging. 支持请求/回复模式的消息传送。
• Easy to use client library to communicate with the DotNetMQ Message Broker. 容易使用客户端库来与DotNetMQ消息代理通信。
• Built-in framework to easily construct RMI services upon message queues. 
• Supports delivering messages to ASP.NET Web Services.
• GUI-based management and monitoring tool.
• Easy to install, manage, and use.
• Written entirely in C# (using .NET Framework 3.5).

　　I preferred to name DotNetMQ as MDS (Message Delivery System) when first creating it. Because it is designed not just to be a message queue, but also as a system that delivers messages directly to applications and an environment that provides a framework to build application services. I called it DotNetMQ since it is entirely developed using .NET and the DotNetMQ name is more memorable. So, it’s original name (and internal project name) is MDS and the applications have many classes with the prefix MDS.

Why a New Message Broker?

The Need for a Message Broker

　　First, I will demonstrate a simple situation where a message broker is needed.

　　In my experiences in business life, I've observed really bad and uncommon asynchronous enterprise application integration solutions. Usually there is an application that runs on a server and performs some tasks and produces data, and then sends the result data to another application on another server. The second application performs other tasks on the data or evaluates the result (the servers are on the same network or connected over the internet). Also, the message data must be persistent. Even if the remote application is not working or the network is not available, the message must be delivered on the first chance.

　　Let’s look at the design in the figure below.

　　Application - 1 and Application - 2 are executable applications (or Windows services) and Sender Service is a Windows service. Application - 1 performs some task, produces data, and calls a Remote Web Service method on Server - B to transmit data. This Web Service inserts data into a database table. Application - 2 periodically checks the table for new incoming data rows and processes them (and deletes them from the table or marks them as processed to not process the same data again).

　　If an error occurs during the Web Service call or while processing data in the Web Service, data must not be lost and must be sent later. However, Application - 1 has other tasks to do, so it can not try to send data again and again. It simply inserts data into a database table. Another Windows service (or a thread in Application - 1, if the application always runs) checks this table periodically and tries to send data to the Web Service until data is successfully sent.

　　This scenario is really reliable (messages are guaranteed to be delivered) but is not an efficient way of communicating between two applications. This solution has some very critical problems:

• It takes a long time to develop (to code).
• Individual coding for all message types (or remote method calls). For a new Web Service method call, you must change all the services, applications, and database tables.
• Almost same software and structures must be developed (or copied and modified) for every similar service.
• Testing and maintenance of too many services/applications/databases after coding.
• Some applications and services periodically check the database even if there is no new message (if the database is not well indexed and optimized, this may consume serious system resources).

　　Message Brokers do all this job and takes all the responsibility to deliver messages to the remote application in the most efficient way. The same application integration using DotNetMQ is shown in the figure below.

　　DotNetMQ is a standalone Windows service that runs on both Server - A and Server - B. Thus, you just need to write code to communicate with DotNetMQ. Using the DotNetMQ Client Library, it is very easy and fast to connect and send/receive messages to/from the DotNetMQ service. Application - 1 prepares the message, sets the destination, and passes the message to the DotNetMQ Broker. DotNetMQ brokers will deliver the message toApplication - 2 in the most efficient and fastest way.

What About Existing Message Brokers

　　It is clear to see that there is a need for Message Brokers to integrate applications. I searched the web, and read books to find a free (and Open Source, if available) Message Broker that is easy to use with .NET. Let's talk about what I found:

• Apache ActiveMQ (http://activemq.apache.org): It is Open Source and implements JMS (Java Message Service is a standard API for messaging in the Java world). It has also a .NET client library. I read a complete book "ActiveMQ in Action" to learn more and I developed some simple applications. Even though I read the book, I did not see an easy and reliable way to construct an ActiveMQ server graph that worked together and routed messages. I also did not see a way to set the destination server for a message. It routes messages automatically but I can not control the routing efficiently. I understood that it is commonly used with Apache Camel (http://camel.apache.org) to achieve common application integration patterns. Apache Camel is also another world to discover, and even worse, it is just for Java. Finally, I think that it is not simple enough to use and especially to configure, monitor, and manage it. So I gave up working on ActiveMQ.
• MSMQ (http://msdn.microsoft.com/en-us/library/ms711472(VS.85).aspx): This is a solution from Microsoftand it is the most suitable framework to use with .NET applications. It is easy to use and learn, and it has tools to monitor queues and messages. It is especially very suitable for asynchronous communication of applications that are running on the same machine or can directly connect to the same machine. But I could not find a built-in solution to construct a graph of MSMQ servers that route messages. Since routing is my first start point, I eliminated this Broker.
• RabbitMQ (http://www.rabbitmq.com): It is developed using the Erlang programming platform (that is developed by Ericsson). You need to install Erlang first. I spent a lot of time to install, configure, and write a sample application. It has a .NET client but I got many errors when trying to develop and run a simple application. It was very hard to install and to make two rabbitMQ Brokers work together on two different servers. After a few days, I gave up because I thought it must not be that hard to learn and to start developing applications.
• OpenAMQ (http://www.openamq.org), ZeroMQ (http://www.zeromq.org): I examined these brokers overallbut I found that I can not easily do what I want to using .NET.
• Others: I also found a few other projects but they have important features missing like routing, persistent messaging, request/reply messaging... etc.

　　You see that there is no Message Broker that is developed entirely in .NET in the list above.

　　From a user perspective, I just want to pass "message data, destination server, and application name" to my localBroker. I am not interested in the rest. It will route a message over the network as many times it requires and delivers the message to my destination application on the destination server. My messaging system must provide this simplicity for me. This was my first start point and I evaluated Message Brokers according to that point. The figure below shows what I want.

　　Application - 1 passes a message to Message Broker in the local server (Server - A):

• Destination server: Server - D
• Destination application: Application - 2
• Message Data: application specific data

　　Server - A has no direct connection to Server - D. So the Message Brokers forward the message over the servers (the message is transmitted through Server - A, Server - B, Server - C, and Server - D sequentially) and the message finally reaches the Message Broker in Server - D to deliver the message to Application - 2. Note that there is another instance of Application - 2 running on Server - E, but it does not receive this message, since the destination server of the message is Server - D.

　　DotNetMQ provides this functionality and simplicity. It finds the best (shortest) path from the source server to the destination server on the graph and forwards the message.