An Introduction to

MQTT

The Message Queuing Telemetry Transport (MQTT) is one of the most popular methods of moving data throughout the factory floor and into the cloud. Much like Modbus TCP, MQTT uses a client/server relationship between devices. One of the biggest differences between MQTT and nearly every other industrial protocol is the publish/subscribe architecture. This architecture allows a message to be published once and go to multiple stake-holding devices, with complete message decoupling between the producer of the data/events and the consumer(s) of those messages and events. Data is arranged by topic in a hierarchy with as many levels of subtopics as needed. Stakeholders can subscribe to a topic or a subtopic. They can also use wildcards to specify topics of interest. One of the biggest challenges MQTT is designed to overcome is the challenge of publishing small pieces of data in volume to many stakeholder devices that are limited by unreliable networks, low bandwidth, and high latency.

The other big difference between MQTT and other industrial protocols is that it utilizes a central server, known as a broker, to facilitate communications between devices. A broker receives the information from the servers and matches the information to consumers by topic subscriptions. The information is distributed to every consumer that matches the topic. If no consumer requires the information, the information is discarded. More information on the broker to follow. MQTT works very well in environments with dynamic communication environments where large quantities of data need to be made available to tens of thousands of devices. The following qualities allow MQTT to be successful:

  • Lightweight code footprint: Devices require no more than a few lines of code to get up and running with MQTT.
  • Speed: MQTT operates in real time and with minimal delays.
  • Minimized data packets: MQTT is energy-efficient, making it perfect for devices that are battery-powered or have little CPU power.
  • Last will and testament: if a client device becomes disconnected unexpectedly, you can configure message instructions to be relayed to all subscriber devices to remedy the situation.

History

MQTT is a protocol that runs over TCP/IP and is used to transport lightweight messages between devices. As manufacturing begins to invest deeper and deeper into the Internet of Things (IoT), the more prevalent MQTT will become on the factory floor. MQTT was created by Andy Stanford-Clark of IBM, and Arlen Nipper of Eurotech. The goal in creating The MQTT protocol was created with the purpose of connecting oil pipelines over unreliable satellite networks. A new protocol was needed—one that was lightweight, bandwidth-efficient, and data agnostic. In 2011, the founders of MQTT donated MQTT to Eclipse to support a project called Paho. The goal of Paho was to provide open-sourced implementations of MQTT in a variety of programming languages for clients. It has since grown to include most new, old, and emerging applications, including the IoT.

MQTT Architecture

To best understand MQTT, the Publish/Subscribe architecture, and the MQTT broker, we must have an understanding of MQTT Topics. Consider this: you have a switch connected to a device, and you wish to use it to change the state of a light which is attached to a different device. Now, with a more traditional protocol, you would probably just connect them directly. But what if you wanted to add new devices with more lights? What if you wanted them to all be triggered from the exact same switch? You are going to need a lot more connections and triggers. In other words, things are about to get overcomplicated very quickly.

Under the Publish/Subscribe system, you can publish the state of your switch under some topic—call it {Switch 01}. Next, you must subscribe your machine, or any number of machines, to {Switch 01}. Whenever a message is published with a matching topic, the broker will relay that information to all subscribing devices. In this case, you will be able to control the state of a light on any number of subscribed devices. You can have any number of different topics with each one having different clients. This flexibility and scalability make it ideal for IoT communications.

MQTT Broker

MQTT’s publish/subscribe model offers many benefits that trump those of traditional poll-and-response protocols. The central server, known as the broker, is responsible for eliminating insecure client connections, scaling the number of device connections, managing client connection states, and reducing network strain. But what is a broker exactly? It is simply software running on a computer, either on the manufacturing site or in the Cloud. You can think of the broker as being almost like a post office within the MQTT network. Say you want to send a birthday card to a friend. When your mailman picks up your card, he doesn’t deliver it directly to your friend. Instead, the letter gets taken to the post office and then delivered to your friend. The same is true for messages within the MQTT network—they must first be sent to the broker before reaching their destination, the client device. The biggest difference here is that instead of sending that published message to a specific address, the message gets forwarded to all devices that are subscribed to that topic. In this model, there is not a direct connection between each client, but rather one outbound connection from the client to the broker. Because of this, every device is interconnected via the broker.

Message Structure

MQTT is not a text-string-based protocol. Rather, it is a binary-based protocol that uses bytes as control elements. MQTT also uses a format that includes both commands and acknowledgement of commands. Some basic commands include Connect, Subscribe, and Publish. The MQTT packet format consists of a two-byte fixed header, a variable header (not always), and a payload (not always).

Fixed Header
The fixed header field consists of the control header and packet length fields. The control header is simply one byte long, while the packet length field ranges between one and four bytes. Within the control header is the eight-bit Control Field. The first four bits determine the command or message type field, and the last four bits act as control flags. The packet length field describes, well, the length of the message being sent. It also specifies how much of the message is useful information.

Variable Length Header and Payload
The variable length header can be thought of as preliminary information that allows you to understand the context of the information that you are looking for. Consider the following scenario: you need to learn the balance of your checking account and you don’t have an electronic means of doing so. You will likely have to call your bank’s customer service number and give them some of your information: your SSN, date of birth, address, etc. This allows the bank to locate your checking account and find the information you are looking for. In this example, the checking account balance is the payload. It’s the meat and potatoes of the message being sent.

Sparkplug

Throughout the years, the MQTT protocol has received many criticisms. Many engineers see the broker as being a point of failure as it is a single source that controls a lot of data. This also makes the broker very attractive to cybercriminals. There was no definition of the payload. Both the clients providing the data to the broker and the clients receiving the data from the broker had to understand the content of the data. This posed problems for manufacturers. Lastly, MQTT was designed for small payloads and applications with many devices, which is the opposite of the situation we have in many industrial applications.

The solution? Sparkplug B, a specification designed around the characteristic details of IoT applications. It defines topics and messages that the MQTT standard doesn’t specify and gives other terminal devices the ability to send data. Sparkplug™ provides an open and freely available specification for how native and gateway devices can communicate bi-directionally within an MQTT Infrastructure in a more structured way. Sparkplug also provides new advantages, including:

  • Support across multiple programming languages and platforms
  • Highly efficient binary coding to replace JSON
  • NDATA/DDATA: only publishes changes in data

With the Sparkplug™ specification, the Topic Namespace is much more well thought out and optimized for more serious application development. For data to be easily useful to other MQTT Client applications that want to consume data values, the Topic Namespace needs to be understood by everyone participating in the data exchange. All MQTT clients using the Sparkplug™ specification will use the following Topic Namespace structure: namespace/group_id/message_type/edge_node_id/[device_id]. Below is an explanation of each component:

  • namespace is either “spAv1.0” or “spBv1.0” (the Sparkplug A and Sparkplug B).
  • group_id is an ASCII string reference to a logical grouping of nodes such as a manufacturing cell or a pipeline segment.
  • message_type element is an ASCII string that defines how the MQTT server processes the message (see state representation below).
  • edge_node_id is the ASCII ID of the MQTT edge device that interfaces to real-world inputs and outputs, as well as gateway device that communicate with a device that has some proprietary or open protocol other than MQTT. An example of the latter is an MQTT edge node that is a Modbus RTU master device.
  • device_id is the ASCII ID of the end device that physically interfaces to the real-world input and outputs. The device ID is optional and not present if the edge node is the actual device interfacing to the real-world inputs and outputs. In the case of where the edge node is a Modbus master, the device ID might be the ID of a Modbus RTU slave on a Modbus RTU network.