In this article by Gastón C. Hillar, author of the book, MQTT Essentials, we will start our journey towards the usage of the preferred IoT publish-subscribe lightweight messaging protocol in diverse IoT solutions combined with mobile apps and Web applications. We will learn how MQTT and its lightweight messaging system work. We will learn MQTT basics, the specific vocabulary for MQTT, and its working modes. We will use different utilities and diagrams to understand the most important concepts related to MQTT. We will:

• Understand convenient scenarios for the MQTT protocol
• Work with the publish-subscribe pattern
• Work with message filtering

(For more resources related to this topic, see here.)

Understanding convenient scenarios for the MQTT protocol

Imagine that we have dozens of different devices that must exchange data between them. These devices have to request data to other devices and the devices that receive the requests must respond with the demanded data. The devices that requested the data must process the data received from the devices that responded with the demanded data.

The devices are IoT (Internet of Things) boards that have dozens of sensors wired to them. We have the following IoT boards with different processing powers: Raspberry Pi 3, Raspberry Pi Model B, Intel Edison, and Intel Joule 570x. Each of these boards has to be able to send and receive data. In addition, we want many mobile devices to be able to send and receive data, some of them running iOS and others Android. We have to work with many programming languages.

We want to send and receive data in near real time through the Internet and we might face some network problems, that is, our wireless networks are somewhat unreliable and we have some high-latency environments. Some devices have low power, many of them are powered by batteries and their resources are scarce. In addition, we must be careful with the network bandwidth usage because some of the devices use metered connections.

A metered connection is a network connection in which we have a limited amount of data usage per month. If we go over this amount of data, we get billed extra charges.

We can use HTTP requests and a build a publish-subscribe model to exchange data between different devices. However, there is a protocol that has been specifically designed to be lighter than the HTTP 1.1 protocol and work better when unreliable networks are involved and connectivity is intermittent. The MQTT (short for MQ Telemetry Transport) is better suited for this scenario in which many devices have to exchange data between themselves in near real time through the Internet and we need to consume the least possible network bandwidth.

The MQTT protocol is an M2M (Machine-to-Machine) and IoT connectivity protocol. MQTT is a lightweight messaging protocol that works with a broker-based publish-subscribe mechanism and runs on top of TCP/IP (Transmission Control Protocol/Internet Protocol). The following diagram shows the MQTT protocol on top of the TCP/IP stack:

The most popular versions of MQTT are 3.1 and 3.1.1. In this article, we will work with MQTT 3.1.1. Whenever we reference MQTT, we are talking about MQTT 3.1.1, that is, the newest version of the protocol. The MQTT 3.1.1 specification has been standardised by the OASIS consortium. In addition, MQTT 3.1.1 became an ISO standard (ISO/IEC 20922) in 2016.

MQTT is lighter than the HTTP 1.1 protocol, and therefore, it is a very interesting option whenever we have to send and receive data in near real time with a publish-subscribe model while requiring the lowest possible footprint. MQTT is very popular in IoT, M2M, and embedded projects, but it is also gaining presence in web applications and mobile apps that require assured messaging and an efficient message distribution. As a summary, MQTT is suitable for the following application domains in which data exchange is required:

• Asset tracking and management
• Automotive telematics
• Chemical detection
• Environment and traffic monitoring
• Field force automation
• Fire and gas testing
• Home automation
• IVI (In-Vehicle Infotainment)
• Medical
• POS (Point of Sale) kiosks
• Railway
• RFID (Radio-Frequency Identification)
• SCADA (Supervisory Control and Data Acquisition)
• Slot machines

As a summary, MQTT was designed to be suitable to support the following typical challenges in IoT, M2M, embedded, and mobile applications:

• Be lightweight to make it possible to transmit high volumes of data without huge overheads
• Distribute minimal packets of data in huge volumes
• Support an event-oriented paradigm with asynchronous bidirectional low latency push delivery of messages
• Easily emit data from one client to many clients
• Make it possible to listen for events whenever they happen (event-oriented architecture)
• Support always-connected and sometimes-connected models
• Publish information over unreliable networks and provide reliable deliveries over fragile connections
• Work very well with battery-powered devices or require low power consumption
• Provide responsiveness to make it possible to achieve near real-time delivery of information
• Offer security and privacy for all the data
• Be able to provide the necessary scalability to distribute data to hundreds of thousands of clients

Working with the publish-subscribe pattern

Before we dive deep into MQTT, we must understand the publish-subscribe pattern, also known as the pub-sub pattern. In the publish-subscribe pattern, a client that publishes a message is decoupled from the other client or clients that receive the message. The clients don’t know about the existence of the other clients. A client can publish messages of a specific type and only the clients that are interested in specific types of messages will receive the published messages.

The publish-subscribe pattern requires a broker, also known as server. All the clients establish a connection with the broker. The client that sends a message through the broker is known as the publisher. The broker filters the incoming messages and distributes them to the clients that are interested in the type of received messages. The clients that register to the broker as interested in specific types of messages are known as subscribers. Hence, both publishers and subscribers establish a connection with the broker.

It is easy to understand how things work with a simple diagram. The following diagram shows one publisher and two subscribers connected to a broker:

A Raspberry Pi 3 board with an altitude sensor wired to it is a publisher that establishes a connection with the broker. An iOS smartphone and an Android tablet are two subscribers that establish a connection with the broker.

The iOS smartphone indicates the broker that it wants to subscribe to all the messages that belong to the sensor1/altitude topic. The Android tablet indicates the same to the broker. Hence, both the iOS smartphone and the Android tablet are subscribed to the sensor1/altitude topic.

A topic is a named logical channel and it is also referred to as a channel or subject. The broker will send publishers only the messages published to topics to which they are subscribed.

The Raspberry Pi 3 board publishes a message with 100 feet as the payload and sensor1/altitude as the topic. The board, that is, the publisher, sends the publish request to the broker.

The data for a message is known as payload. A message includes the topic to which it belongs and the payload.

The broker distributes the message to the two clients that are subscribed to the sensor1/altitude topic: the iOS smartphone and the Android tablet.

Publishers and subscribers are decoupled in space because they don’t know each other. Publishers and subscribers don’t have to run at the same time. The publisher can publish a message and the subscriber can receive it later. In addition, the publish operation isn’t synchronized with the receive operation. A publisher requests the broker to publish a message and the different clients that have subscribed to the appropriate topic can receive the message at different times. The publisher can perform asynchronous requests to avoid being blocked until the clients receive the messages. However, it is also possible to perform a synchronous request to the broker and to continue the execution only after the request was successful. In most cases, we will want to take advantage of asynchronous requests.

A publisher that requires sending a message to hundreds of clients can do it with a single publish operation to a broker. The broker is responsible of sending the published message to all the clients that have subscribed to the appropriate topic. Because publishers and subscribers are decoupled, the publisher doesn’t know whether there is any subscriber that is going to listen to the messages it is going to send. Hence, sometimes it is necessary to make the subscriber become a publisher too and to publish a message indicating that it has received and processed a message. The specific requirements depend on the kind of solution we are building. MQTT offers many features that make our lives easier in many of the scenarios we have been analyzing.

Working with message filtering

The broker has to make sure that subscribers only receive the messages they are interested in. It is possible to filter messages based on different criteria in a publish-subscribe pattern. We will focus on analyzing topic-based filtering, also known as subject-based filtering.

Consider that each message belongs to a topic. When a publisher requests the broker to publish a message, it must specify both the topic and the message. The broker receives the message and delivers it to all the subscribers that have subscribed to the topic to which the message belongs.

The broker doesn’t need to check the payload for the message to deliver it to the corresponding subscribers, it just needs to check the topic for each message that has arrived and needs to be filtered before publishing it to the corresponding subscribers.

A subscriber can subscribe to more than one topic. In this case, the broker has to make sure that the subscriber receives the messages that belong to all the topics to which it has subscribed. It is easy to understand how things work with another simple diagram. The following diagram shows two future publishers that haven’t published any message yet, a broker and two subscribers connected to the broker:

A Raspberry Pi 3 board with an altitude sensor wired to it and an Intel Edison board with a temperature sensor wired to it will be two publishers. An iOS smartphone and an Android tablet are two subscribers that establish a connection to the broker.

The iOS smartphone indicates the broker that it wants to subscribe to all the messages that belong to the sensor1/altitude topic. The Android tablet indicates the broker that it wants to subscribe to all the messages that belong to any of the following two topics: sensor1/altitude and sensor42/temperature. Hence, the Android tablet is subscribed to two topics while the iOS smartphone is subscribed to just one topic.

The following diagram shows what happens after the two publishers connect and publish messages to different topics through the broker:

The Raspberry Pi 3 board publishes a message with 120 feet as the payload and sensor1/altitude as the topic. The board, that is, the publisher, sends the publish request to the broker. The broker distributes the message to the two clients that are subscribed to the sensor1/altitude topic: the iOS smartphone and the Android tablet.

The Intel Edison board publishes a message with 75 F as the payload and sensor42/temperature as the topic. The board, that is, the publisher, sends the publish request to the broker. The broker distributes the message to the only client that is subscribed to the sensor42/temperature topic: the Android Tablet. Thus, the Android tablet receives two messages.

Summary

In this article, we started our journey towards the MQTT protocol. We understood convenient scenarios for this protocol, the details of the publish-subscribe pattern and message filtering. We learned basic concepts related to MQTT and understood the different components: clients, brokers, and connections.

Resources for Article:

---

Further resources on this subject:

• All About the Protocol [article]
• Analyzing Transport Layer Protocols [article]
• IoT and Decision Science [article]

---