MQTT Protocol Description

Source

Table of contents:

• What is MQTT?
• Features of the MQTT protocol, MQTT description
• Topics semantics
• Message structure
• Quality of service in the MQTT protocol (QoS)
• Data protection
• Equipment for IIoT with MQTT support

With the development of industry, the number of devices that need to be monitored and receive various data from them is increasing. To solve the problems of the interaction of a large number of devices and the problems of integrating devices into one network, the concept of the Internet of Things - when devices are combined for some reason into one network, then several similar networks are combined into another big net and so on.

Devices in such networks interact with each other by means of various interfaces and data transfer protocols. Since we are talking about the industrial application of the IoT concept, which should use industrial equipment with its own protocols and hardware, we turn to the concept of IIoT (Industrial Internet of Things).

To communicate with each other, devices use various industrial protocols, one of the most popular protocols for this purpose is MQTT.

What is MQTT?

MQTT or Message Queue Telemetry Transport is a lightweight, compact and open data exchange protocol designed to transfer data to remote locations where small code size is required and there are bandwidth limitations. The above advantages allow it to be used in M2M (Machine-Machine Interaction) and IIoT (Industrial Internet of Things) systems.

There is also a version of the MQTT-SN protocol (MQTT for Sensor Networks), previously known as MQTT-S, which is designed for embedded wireless devices without TCP / IP support, for example, Zigbee.

MQTT protocol features

The main features of the MQTT protocol:

• Asynchronous protocol
• Compact messages
• Work in conditions of unstable communication on the data line
• Support for multiple levels of quality of service (QoS)
• Easy integration of new devices

The MQTT protocol works at the application level over TCP / IP and uses the default 1883 port (8883 when connected via SSL).

The exchange of messages in the MQTT protocol is carried out between the client, which can be the publisher or subscriber of messages, and the broker of messages (for example, Mosquitto MQTT).

The publisher sends data to the MQTT broker, indicating in the message a specific topic.
Subscribers may receive different data from multiple publishers, depending on the subscription to the relevant topics.

MQTT devices use certain types of messages to interact with the broker, below are the main ones:

• Connect – establish a connection with a broker
• Disconnect – disconnect from broker
• Publish – опубликовать данные в топик на брокере
• Subscribe – подписаться на топик на брокере
• Unsubscribe – отписаться от топика

Scheme of simple interaction between
the subscriber, publisher and broker

Topics semantics

Topics are UTF-8 encoded characters. The hierarchical structure of topics has a “tree” format, which simplifies their organization and access to data. Topics consist of one or several levels, which are separated by the symbol “/”.

An example of a topic in which a temperature sensor located in a bedroom publishes data to a broker:
/ home / living-space / living-room1 / temperature

A subscriber can also receive data from several topics at once, for this there is a wildcard. They are of two types: single-level and multi-level. For a simpler understanding, consider each of them in the examples:

• Sibling wildcard. The “+” symbol is used to apply it.
  For example, we need to obtain temperature data in all bedrooms:

  /home/living-space/+/temperature
  

  As a result, we obtain data from the topics:

  /home/living-space/living-room1/temperature

  /home/living-space/living-room2/temperature

  /home/living-space/living-room3/temperature

• Layered wildcard. The symbol “#” is used to apply it.

  For example, to obtain data from various sensors of all bedrooms in a house:
  

  /home/living-space/#

  As a result, we obtain data from the topics:

  /home/living-space/living-room1/temperature

  /home/living-space/living-room1/light1

  /home/living-space/living-room1/light2

  /home/living-space/living-room1/humidity

  /home/living-space/living-room2/temperature

  /home/living-space/living-room2/light1

  …

Message structure

MQTT message consists of several parts:

• Fixed header (present for all messages)
• Variable header (only present in certain messages)
• Data, "load" (only present in certain messages)

Fixed header

Message Type – for example: CONNECT, SUBSCRIBE, PUBLISH and others.

Flags specific to each MQTT packet – these 4 bits are reserved for auxiliary  flags, the presence and status of which depends on the type of message. 

Remaining Length – represent the length of the current message (variable header + data), can take from 1 to 4 bytes.

In total, there are 15 message types in the MQTT protocol:

*C – Client, **S – Server

Flags

Four high bits of the first byte of the fixed header are reserved for special flags:

DUP – the duplicate flag is set when the client or MQTT broker resends the packet (used in the types PUBLISH, SUBSCRIBE, UNSUBSCRIBE, PUBREL). When checked, the variable header should contain a Message ID (Message ID)

QoS – Quality of Service (0,1,2)

RETAIN – when publishing data with the retain flag set, the broker will save it. The next time you subscribe to this topic, the broker will immediately send a message with this flag. Used only in messages with type PUBLISH.

Variable title

The variable title is contained in some headers.

It contains the following data:

• Packet identifier – present in all message types, except: CONNECT, CONNACK, PUBLISH(с QoS <1), PINGREQ, PINGRESP, DISCONNECT
• Protocol name – only in messages of type CONNECT
• Protocol version – only in messages of type CONNECT
• Connect flags – flags indicating the client's behavior when connecting

User name – if this flag is present, the user name must be specified in the "load" (used for client authentication)

Password – if this flag is present, the password must be specified in the “load” (used for client authentication)

Will Retain – when set to 1, the broker stores Will Message.

Will QoS – quality of service for Will Message, with the Will Flag flag set, Will QoS and Will retain required.

Will Flag - when the flag is set, after the client disconnects from the broker without sending the DISCONNECT command (in cases of unpredictable communication failure, etc.), the broker will notify all connected clients via the so-called Will Message..

Clean Session – clean the session. When set to “0,” the broker will save the session, all client subscriptions, and also pass on to him all messages with QoS1 and QoS2 that were received by the broker during the client’s disconnection at the next connection. Accordingly, when “1” is set, when reconnecting, the client will need to re-subscribe to topics.

• Session Present – used in a message with a CONNACK type. If the broker accepts a connection with Clean Session = 1, it must set “0” to the Session Present (SP) bit. If the broker accepts a connection with Clean Session = 0, then the value of the SP bit depends on whether the broker has previously saved the session with this client (if so, 1 is set in SP and vice versa). This parameter allows the client to determine whether the previous session was saved by the broker.
• Connect Return code – if for some reason the broker cannot accept a correctly formed CONNECT packet from the client, then in the second byte of the CONNACK packet he should set the corresponding value from the list below:

  Value	Return value	Description
  0	0x00	Connection Accepted
  1	0x01	Connection Refused, unacceptable protocol version
  2	0x02	Connection Refused, identifier rejected
  3	0x03	Connection Refused, Server unavailable
  4	0x04	Connection Refused, bad user name or password
  5	0x05	Connection Refused, not authorized
  6-255		Reserved

• Topic Name 

Data

The content and format of the data transmitted in MQTT messages are defined in the application. The data size can be calculated by subtracting the length of the variable header from Remaining Length.

Quality of service in the MQTT protocol (QoS)

MQTT supports three levels of quality of service (QoS) in the transmission of messages.

QoS 0 At most once.
At this level, the publisher sends a message to the broker once and does not wait for confirmation from him, that is, sent and forgotten.

QoS 1 At least once.
This level ensures that the message will be delivered exactly to the broker, but there is a possibility of duplication of messages from the publisher. After receiving a duplicate message, the broker again sends this message to subscribers, and the publisher again sends confirmation of receipt of the message. If the publisher has not received a PUBACK message from the broker, he re-sends this package, while the “1” is set in the DUP.

QoS 2 Exactly once.
At this level, delivery of messages to the subscriber is guaranteed and possible duplication of sent messages is excluded.

The publisher sends a message to the broker. This message indicates a unique Packet ID, QoS = 2 and DUP = 0. The publisher keeps the message unconfirmed until it receives a response from the broker PUBREC. The broker responds with a PUBREC message that contains the same Packet ID. After receiving it, the publisher sends a PUBREL with the same Packet ID. Before the broker receives PUBREL, he must keep a copy of the message at home. Upon receiving PUBREL, it deletes a copy of the message and sends a message to the publisher that the transaction is complete.

Data protection

To ensure security, the following protection methods are implemented in the MQTT protocol:

• Client authentication. A CONNECT package can contain USERNAME and PASSWORD fields.
  When implementing a broker, you can use these fields for client authentication.
• Client Access Control through Client ID
• Connect to a broker via TLS / SSL