A long time ago a colleague of mine, Bhaskar Apparajuvenkata, and I used WCF to implement a publish-subscribe system on our local network. (That was blogged here.) Ok it was less than 5 years ago but given the pace of change since then it feels very antiquated. At the time we were having a hard time convincing our infrastructure team to stand up MSMQ for us. This was before the software-as-a-service movement took hold. Back in those days something like a messaging or service bus framework was seen as an enterprise project. You had to have a project sponsor, huge timelines, and of course a big budget. We were a small development team and our needs were modest so at the time we rolled our own.
Fast forward to today. The system works great in production and we can't imagine life without some sort of message-based architecture. But we're feeling some pain. WCF is overkill. The publisher is a WCF service and subscribers must also be WCF services. We have about a dozen top-heavy subscribers at this point. Some developers just aren't comfortable with WCF configuration or debugging. And testing is tricky. Also our events and subscriber endpoints are stored in a database requiring additional maintenance on our part. So now we're looking at our options for replacing it.
Here are my three main requirements for a replacement solution: (1) simple and lightweight; (2) avoid the need to store and maintain subscriber information; and (3) avoid subscriber long-polling.
Our first obvious option was Azure Service Bus since we already use it with a third-party partner to store transactions for later processing. It's durable, scaleable, simple, and has a rich .NET API. But we have one big problem. Our subscribers are on-prem and because of the firewall we'd have to do long-polling over port 80 to listen for new messages. Azure does provide a broker option called a "service bus relay" that addresses this problem; however it requires WCF bindings to work. That would be déjà vu all over again.
So next up is RabbitMQ. Fast, lightweight, and popular. In just a few hours my colleague stood up a local Ubuntu LTS VM with a RabbitMQ installation. We're doing pubsub so we created a single fanout exchange called "test-exchange." To keep it simple the test-exchange broadcasts its messages to a single queue called "test-queue." For this spike the producer (publisher) is a .NET console app that does nothing but publish the current time. It requires the RabbitMQ.Client nuget package.
The above publisher code is really a snippet that can be placed in any application that raises events for other applications to handle. This is the essence of message-driven architecture of course. The real problem for us to solve was the subscriber. Most blogs and RabbitMQ tutorials all show console apps. Not very helpful in a production scenario. I thought about it being a Windows Service. But they're a pain to debug and deploy.
How about ASP.NET RESTful Web API? In our shop we're leaning in a microservices direction and for us that means smaller Web API apps. They're easy to write, debug, and deploy. Devs understand HTTP and no complicated WCF configuration required. Here's what I did and it's remarkably simple...
I spun up a new ASP.NET Web API solution in Visual Studio. Then I added the RabbitMQ.Client nuget package. I added a folder named Bus and added this class to it:
The plumbing is very similar to the producer code. But notice the ConnectionFactory has two additional properties to recover from a network failure. This is very important to configure because something will go wrong and connections will be interrupted. In my testing I simulated a network outage on the subscriber server itself. We also took down the whole RabbitMQ server and brought it back up. Through both outages the connection was restablished with no problem. The Received event is handled by the delegate function ConsumerOnReceived. It just writes the message body out to the debug output window. In a real production app you'd want to discover what message you received and handle it. This is where a Gang of Four Builder pattern would come in handy. [Update: See Part 2 in this series for a real-world messaging example.]
With the MessageListener class in place we need a way to start it up and keep it running through the application lifecycle of the ASP.NET app. When the app pool gets recycled or if the AppDomain goes down for any reason we need to start it up again. And we want the MessageListener to tear down its connection to the queue gracefully. The bootstrap code to do this gets added to Global.asax.cs:
Obviously you would get those ConnectionFactory values out of the code and into the Web.config file. And you'd want to set noAck to false and make sure to reply back to RabbitMQ with an ACK so that no messages are lost. But otherwise that's all there is to it.
In a future blog post I'd like to go a little deeper with this spike. I'd like to add message payloads that are a little more interesting than the current time. And I'd like to show how to use a Gang of Four Builder pattern in the ConsumerOnReceived delegate function to handle processing of different kinds of messages. But for now this will have to do.