Discover more from Fogplug by Thor Ingham
MQ died and APIs rule the world
Nice title, misleading on purpose. A colleague of mine had a great way of describing when you should use message queueing (MQ) and not an Application Programming Interface (API).
“If your data matters, MQ!” Short and simple,
Thank you to Lars Dehli at Avella AS for that one.
One question that keeps coming from customers is that they want APIs and only APIs. I understand why, 99% of the marketing is about APIs these days.
Lets start by defining two types of APIs. The way I see it.
One type of API is the system API, the API that comes with your application so that you can interact with it. Most Applications these days have an API, the API could be a Rest interface, web service or even a drop and forget file folder. Here is the thing, an API is also an MQ endpoint which is accessed by a client either using an MQ client or even Rest. IBM MQ now supports rest.
The other type of APIs is the product API. These APIs are gives access to something that also has business value and a business description behind it. APIs are products require API management. API management usually consists of an API gateway, the runtime for you API. The Gateway typically handles security, routing, throttling and more. There will be a portal, a CMS that allows you describe and socialise your APIS.
When I clients says they want APIs for everything they are most of time talking about the second type of API. What enterprises want to be able to do is publish digital assets, or assets digitally (the first a digital first product, the latter is modernisation) as a modern API.
Then we have MQ. MQ is a messaging system, it is a way to transport messages between applications and between Microservices that make up an application. Is is also a messaging style that often is a good idea to have behind any API, even the product APIs because they provide something a HTTP based call will not. APIs that are Rest or web services typically go over HTTP.
If we look at MQ I have a list of reasons to use it.
MQ provides a pub/sub interface where-as HTTP is point-to-point.
The APIs you have need to be available 24/7 - you can not shot the down, well you do not want to. However there is no API with a service or application handling the requests. You need to be able to move and maintain these backend services. One way to make this easier is to put a queue in between. This allows messages to buffered until the receiver is able to process the message. You can even accept a request, tell your client, sorry we will process this later because we have some maintance going on, do not worry - your message is in a queue. We refer to this as temporal loose coupling.
A process that takes time to be completed is better suited for MQ as you can send a request and have a separate thread listening for responses.
Loose coupling, again because it is so often forgotten about. With HTTP you need retries, if your client retries every second or even more, this could crash your system. Hence why the API gateway needs to support throttling.
MQ allows you set a message priority.
MQ allows you build an define a MQ landscape with secure messaging between data centers (and clouds) so that your developers only need to think about business value when writing code. They can trust the MQ infrastructure in place to be secured and robust.
XA transactions – MQ is fully XA compliant – HTTP is not.
Fault tolerance – MQ messages survive server or network failures – HTTP does not.
MQ has the option of guaranteed delivery of messages and even exactly once only, http does not.
MQ provides the ability to do message segmentation and message grouping for large messages – HTTP does not have that ability as it treats each transaction separately.
To summarise, it could be that you want a public facing APIs. I guarantee you that your API will be superior in regards to security and robustness if you architect for MQ as well. To answer when to use MQ and have your clients use an MQ client, it depends - if it is not a public product API. Well then it is what Lars Dehli said, “If your data matters, MQ!”. All in all a compelling case for considering MQ in addition to APIs.