🥇RabbitMQ. Part 1. Introduction. Erlang, AMQP

Good day, Habr! I want to share a tutorial-guide of the knowledge I've managed to gather on RabbitMQ and condense into short recommendations and conclusions.

RabbitMQ. Part 1. Introduction. Erlang, AMQP and RPC
RabbitMQ. Part 2. Understanding Exchanges
RabbitMQ. Part 3. Understanding Queues and Bindings
RabbitMQ. Part 4. Understanding what messages and frames are
RabbitMQ. Part 5. Performance of Message Publishing and Consumption
RabbitMQ. Part 6. Overview of Federation and Shovel Modules
RabbitMQ. Part 7. Detailed about Connection and Channel
RabbitMQ. Part 8. RabbitMQ in .NET
RabbitMQ. Part 9. Monitoring

Briefly about AMQP

AMQP (Advanced Message Queuing Protocol) is an open protocol for message transmission between system components. The main idea is that individual subsystems (or independent applications) can exchange messages in various ways through the AMQP broker, which performs routing, possibly guarantees delivery, distributes data streams, and subscribes to the desired types of messages.

Protocol AMQP introduces three concepts:

exchange (exchange point) — where messages are sent. The exchange point distributes messages to one or more queues. It routes messages to a queue based on created bindings (binding) between it and the queue
queue (queue) — a data structure in disk or RAM that stores references to messages and provides copies of messages consumers (to consumers)
binding (binding) — a rule that informs the exchange point which queues these messages should go to.

The protocol operates on top of TCP/IP.

Briefly about Erlang

The source code of the project is located in the repository on GitHub. The architecture RabbitMQ-server is based on Erlang and BEAM.

Erlang developed by Ericsson in the mid-1980s as a distributed, fault-tolerant real-time system for applications requiring 99.999% uptime. Erlang is used in various industries and modern applications, for example in WhatsApp. More details can be found in the article WhatsApp Architecture, which Facebook bought for $19 billion

Briefly about RabbitMQ

RabbitMQ – is an open-source message broker. It routes messages according to the fundamental principles of the protocol AMQP described in the specification. RabbitMQ implements and complements the protocol. AMQP.

The main idea of the message exchange model in RabbitMQ is that the producer (the publisher) does not send messages directly to the queue. In fact, quite often the publisher doesn't even know if a message will be delivered to any queue at all.

Instead, the publisher can only send messages to an exchange. On one side, the exchange receives messages from publishers, and on the other, it sends them to queues. The exchange must know exactly what to do with the incoming message. Should it be added to a specific queue? Should it be added to several queues? Or should the message be ignored?

The workflow can be described as follows: RabbitMQ The publisher sends a message to a specific exchange.

The exchange, upon receiving the message, routes it to one or more queues according to the binding rules between itself and the queue.
The queue holds a reference to this message. The message itself is stored in memory or on disk.
As soon as the consumer is ready to retrieve a message from the queue, the server creates a copy of the message according to the reference and sends it.
The consumer receives the message and sends an acknowledgment to the broker.
The broker, upon receiving the acknowledgment, deletes the copy of the message from the queue. It then removes it from memory and disk.
RPC

RPC

Process RPC (remote procedure call) underlies almost all interactions with the core. RabbitMQFor example, initial discussions of the client's terms with RabbitMQ, demonstrates a certain process. RPCOnce this sequence is completed, RabbitMQ will be ready to accept requests from the client.

Also, in the specification, AMQP both the client and server can invoke commands. This means that the client expects interaction with the server. Commands are classes and methods. For example, Connection.Start – a method call. Start class Connection.

Connections and Channels

Channels are used for such information exchange between the client and server. Channels.Channels are created within a specific connection.Each channel is isolated from other channels. In a synchronous scenario, it's not possible to execute the next command until a response is received.

To be able to send commands in parallel, multiple channels must be opened. Each channel creates a separate Erlang process. A single connection can have multiple channels (multiplexing). Each channel has certain structures and objects in memory. Therefore, the more channels there are within a connection, the more memory RabbitMQ uses to manage such a connection.

A simple example of creating a connection and a channel using RabbitMQ.Client:

// ...
private void TryConnect()
{
    var factory = new ConnectionFactory() 
    {
        HostName = "host_name",
        UserName = "user_name",
        Password = "p@ssword",
        // Включение автоматичекого восстановления
        // соединения после сбоев сети 
        AutomaticRecoveryEnabled = true
    };
    _connection = factory.CreateConnection();
}
// ...
public void CreateChanel()
{
    _channel = _connection.CreateModel();
    // other options 
}

Opening a new connection for each operation is strongly discouraged, as this will lead to significant costs. Channels should also be persistent, but many protocol errors can lead to channel closures, so the channel's lifespan may be shorter than that of the connection.

Where is RabbitMQ used?

In the context of microservices, the protocol AMQP and its implementation in RabbitMQ are often used for asynchronous communication between services.

In the context of IIOT the protocol AMQP and its implementation in RabbitMQ is used for data exchange between servers (server-to-server). The RabbitMQ MQTT Plugin is utilized as an implementation of the protocol MQTT for transmitting data between sensors and servers in low-bandwidth environments with high latency (a complete list of supported protocols can be found on the project site).

In the next article, we will delve deeper into Exchanges.