What are Message Brokers and why we need them? 🕵🏻‍♀️

What is a Message Broker ➾

Let's discuss what is the message broker and its features and the use cases.

A message broker is a software that permits systems, applications, and services to communicate with one another. This is done by converting messages between formal messaging protocols. The message broker permits inter-dependent services to “talk” with each other directly, even if they were written in different kinds of programming languages or executed on different platforms.

Message brokers have the ability to store, validate, route, and deliver messages to the correct as well as for the suitable destinations. They distribute as intermediaries between other applications, permitting the senders to provide messages without even knowing how many or where the receivers are.

So the message brokers can do below four things:

1. It has the ability to divide the publisher and consumer.

2. It is able to store the messages.

3. It has the ability to route messages.

4. It has the ability to check and organize messages.

There are 2 Types of Message Broker models

Point-to-point messaging

Ref:https://docs.oracle.com/cd/E19316-01/820-6424/aerbj/index.html

This is the pattern used in Point-to-point messaging with a one-to-one relationship between the sender and receiver. Here what actually happens is The message which the sender sent to the queue; is retrieved by the consumer from the queue and sends acceptance that the message was received. Each message in the queue is sent to only one recipient and is consumed only one time.

Publish/subscribe messaging

Ref:https://docs.oracle.com/cd/E19316-01/820-6424/aerbj/index.html

According to the Publish/subscribe messaging model, the producer is called a publisher and the consumer is called a subscriber.

Here, the producers send messages on a topic. So here messages from one or many publishers can be received by many subscribers.

Let's discuss some examples of message brokers

🟡 Apache Kafka

Apache Kafka is an event streaming platform that use to collect, process, store, and integrate data. This is is more mature & stable distributed platform. Also, we can utilize Apache Kafka with a wide scope of systems such as microservices, web applications, desktop applications, NoSQL, Oracle, SFDC..etc. Mostly used by most companies for banking, retail, eCommerce.. etc.

Below are the 5 main components of Kafka Message Broker.

Consumer API, Producer API, Connector API, Streams API, and Admin API.

Below are some use cases of Kafka:

Large scale massage processing

Complex analysis

Hybrid messaging db

Scalable data structure

When discussing the working principles of Kafka. There are 2 main patterns of messaging. They are:

1. Queuing
2. Publish-subscribe

Pros and Cons of Kafka:

🟡 RabbitMQ

RabbitMQ is the most extensively deployed open-source message broker. This message broker supports both the Point-to-point messaging and Publish /subscribe messaging mechanisms.

Below are some key features of RabbitMQ:

✶ Message ordering not supported.

✶ Reliable delivery.

✶ Has routing capabilities.

✶ RabbitMq is a queue, so messages are done away with once consumed and the acknowledgment is sent.

Pros:

• Fit for many programming languages
• Easy to start using
• Easy to deploy
• Guarantees delivery of messages
• Fir for many messaging protocols
• Can be used in different types of operating systems/cloud systems

Cons:

• By default it is Non-transactional
• Have problems with processing big amounts of data

🟡 ActiveMQ

ActiveMQ is more in the RabbitMQ group than Kafka. This is an open-source protocol that was developed by Apache. The main function here is to send messages in between different applications. ActiveMQ supports some enterprise features too.

Pros:

• Dynamic queue creation.
• Is configurable through XML.
• The product is actively maintained.
• Routing

Cons:

• Messages must be pass to either queue.

🟡 Redis

Redis is somewhat different from the other message brokers we talked about earlier. At its core, Redis is an in-memory data storage that can be utilized as either a high-performance key-value store. ( or as a message broker). This is like a database that can be utilized as a message broker.

Also, Redis doesn’t have persistency and Redis is more perfect for real-time data processing. Another important fact is that the Redis message broker supports only transient messages.

The most important thing is that Redis doesn't slow down when increasing the number of consumers. This message broker supports only the Publish/subscribe messaging mechanism. Also, this doesn’t guarantee the delivery of each message.

✶Redis is super FAST.

✶Simple.

✶Easy to use.

✶Uses Redis Hashing as the hashing mechanism.

References

Publish/Subscribe Messaging (Sun Java System Message Queue 4.3 Technical Overview)

Kafka versus traditional messaging frameworks - WHISHWORKS

[1]”What is a Message Broker?”, TIBCO Software, 2021. [Online]. Available: https://www.tibco.com/reference-center/what-is-a-message-broker.

[2]”Kafka versus traditional messaging frameworks — WHISHWORKS”, WHISHWORKS, 2021. [Online]. Available: https://www.whishworks.com/blog/data-analytics/big-data-messaging-frameworks-compared/. [Accessed: 15- May- 2021]