MQTT adapter
The MQTT protocol adapter exposes an MQTT topic hierarchy for publishing messages and events to the Telemetry and Event endpoints of the device connectivity layer.
The adapter does not behave like a general purpose MQTT broker.
Therefore the following applies to the adapter:
Supports MQTT 3.1.1 only.
Does not maintain session state for clients.
Therefore setting the clean session flag on the client side does not have an impact on the adapter.
The adapter will set the session present to 0 in the CONNACK packet in any case.Ignores any Will contained in the CONNECT packet.
Only supports the topic names mentioned in the sections below.
Custom topics are not supported.Does not support retaining messages.
However, if the retain flag is present in a message, then a x-opt-retain property is added to the downstream message and may be mapped in the header mappings of the connection (managed in the digital twin layer) .
The application can then decide to react upon the presence of this annotation.
Table of contents
Establishing the connection
The MQTT adapter requires clients to authenticate during connection establishment. The adapter supports both username/password based authentication as well as client certificate based authentication. In order to do so, clients need to provide a username and a password in the MQTT CONNECT packet or a client certificate. The username must have the format auth-id@tenant-id.
The server certificate of the device connectivity layer is issued by Let’s Encrypt. Some devices directly trust the Let’s Encrypt root certificate, in such case downloading the server certificate is not necessary. If that is not the case, you need to import the Let’s Encrypt certificate of the device connectivity layer in your trust store to establish a TLS connection to the MQTT adapter.
For the examples below mosquitto_pub is used. mosquitto_pub needs the server certificate to establish a TLS connection to the device connectivity layer.
The device connectivity layer uses Let's Encrypt ISRG Root X1 certificate. The certificate can be downloaded as follows:
curl -o ISRGRootX1.crt https:
//letsencrypt.org/certs/isrgrootx1.pem
Inactivity timeout
The client should provide a Keep Alive value of less than 60s in the MQTT CONNECT packet to ensure that connection issues can be detected. Connected devices which are neither sending nor receiving data will be disconnected after one and a half times the Keep Alive time period, or after 60s, whichever comes first. To keep the connection open a device must send MQTT PINGREQ packages if needed to indicate to the server that the device is still active.
Note that if a device is subscribed to receive commands and the MQTT connection fails, a subsequent successful new connection and command subscription by the client may still result in commands not getting delivered until the Keep Alive time period has elapsed and the server detects the connection failure. This should be taken into consideration when choosing the Keep Alive value.
Communication patterns
Publishing telemetry data
The MQTT adapter supports publishing of telemetry data by means of MQTT PUBLISH packets using QoS 0 or QoS 1. The devices can optionally indicate the content type of the payload by setting the content-type property explicitly in the property-bag. The details regarding configuring the property-bag for telemetry messages can be found in Hono Documentation.
Publishing events
The MQTT adapter supports publishing of events by means of MQTT PUBLISH packets using QoS 1 only. The adapter will send an MQTT PUBACK packet to the client once the event has been stored in the messaging component of the device connectivity layer. The devices can optionally indicate a time-to-live duration for event messages and the content type of the payload by setting the hono-ttl and content-type properties explicitly in the property-bag. The details on configuring the property-bag for event messages can be found in Hono Documentation.
Command & control
The MQTT adapter supports receiving commands by means of MQTT SUBSCRIBE. Devices can subscribe to commands with QoS 0 or QoS 1. The responses to commands can being published using either QoS 0 or QoS 1.
All above mentioned communication patterns can be used by devices connecting directly to the MQTT protocol adapter or by gateways connecting to the MQTT protocol adapter and acting on behalf of the actual devices. See Gateway mode for a more detailed description.
Devices connected directly to MQTT adapter
Publishing telemetry data
Topic: telemetry
Authentication: required with credentials of device
Payload:
(required) Arbitrary payload
Example:
Upload a JSON string for device:
mosquitto_pub -h mqtt.bosch-iot-hub.com -p
8883
-u {auth-id}@{tenant-id} -P {password} -t telemetry -m
'{"temp": 5}'
--cafile ISRGRootX1.crt
Publishing events
Topic: event
Authentication: required with credentials of device
Payload:
(required) Arbitrary payload
Example:
Upload a JSON string for device:
mosquitto_pub -h mqtt.bosch-iot-hub.com -p
8883
-u {auth-id}@{tenant-id} -P {password} -t event -m
'{"alarm": 1}'
-q
1
--cafile ISRGRootX1.crt
Receiving commands
Topic Filter: command///req/#
The actual topic is structured as follows:
for one-way commands: command///req//{command-subject}
for request/response commands command///req/{req-id}/{command-subject}
Parameters:
command-subject: The command subject (e.g. the name of the command to execute)
QoS: Devices can subscribe with QoS 1 or QoS 0.
Authentication: required with credentials of device.
SUBACK response will contain "Failure" (0x80) value if subscription failed because of either an invalid topic filter or a (temporary) server error. SUBSCRIBE retries should be applied if needed.
The payload of the message contains the command payload.
Subscribe to command topic:
mosquitto_sub -d -h mqtt.bosch-iot-hub.com -p
8883
-u {auth-id}@{tenant-id} -P {secret} -k
30
--cafile ISRGRootX1.crt -t command
///req/#
Sending a response to a command
Topic: command///res/{req-id}/{status-code}
req-id: the unique identifier of the command execution request
status-code: a HTTP status code indicating the outcome of executing the command.
Authentication: required with credentials of device
Payload:
(required) Arbitrary command payload
Send back command response:
mosquitto_pub -d -h mqtt.bosch-iot-hub.com -p
8883
-u {auth-id}@{tenant-id} -P {secret} --cafile ISRGRootX1.crt -t command
///res/{req-id}/200 -m '{"greetingDurationInSec": 23}'
Gateways connected to MQTT adapter
Publishing telemetry data
Topic: telemetry/{tenant-id}/{device-id}
Authentication: required with credentials of gateway
Payload:
(required) Arbitrary payload
In Gateway Mode {device-id} is the ID of the actual device while credentials are the credentials of the gateway.
Example:
Upload a JSON string for device through gateway:
mosquitto_pub -h mqtt.bosch-iot-hub.com -p
8883
-u {auth-id}@{tenant-id} -P {password} -t telemetry/{tenant-id}/{device-id} -m
'{"temp": 5}'
--cafile ISRGRootX1.crt
Publishing events
Topic: event/{tenant-id}/{device-id}
Authentication: required with credentials of gateway
Payload:
(required) Arbitrary payload
In Gateway Mode {device-id} is the ID of the actual device while credentials are the credentials of the gateway.
Example:
Upload a JSON string for device through gateway:
mosquitto_pub -h mqtt.bosch-iot-hub.com -p
8883
-u {auth-id}@{tenant-id} -P {password} -t event/{tenant-id}/{device-id} -m
'{"alarm": 5}'
-q
1
--cafile ISRGRootX1.crt
Receiving commands
The gateway may receive commands for all devices in whose behalf it acts using topic filter command//+/req/# or for a specific device using topic filter command//${device-id}/req//{command-subject}.
Topic Filter to subscribe to commands for all devices in whose behalf a gateway acts: command//+/req/#
The actual topic is structured as follows:
for one-way commands for all devices: command//+/req//{command-subject}
for request/response commands for all devices: command//+/req/{req-id}/{command-subject}
Topic Filter to subscribe only to commands for a specific device, an authenticated gateway uses the topic filter command//${device-id}/req/#
The actual topic is structured as follows:
for one-way commands for a specific device: command//${device-id}/req//{command-subject}
for request/response commands for a specific device: command//${device-id}/req/{req-id}/{command-subject}
QoS: Devices can subscribe with QoS 1 or QoS 0.
Authentication: required with credentials of gateway.
SUBACK response will contain "Failure" (0x80) value if subscription failed because of either an invalid topic filter or a (temporary) server error. SUBSCRIBE retries should be applied if needed.
The payload of the message contains the command payload.
Subscribe to command topic:
A subscription to commands for all devices that a gateway acts on behalf of looks like this:
mosquitto_sub -d -h mqtt.bosch-iot-hub.com -p
8883
-u {auth-id}@{tenant-id} -P {secret} -k
30
--cafile ISRGRootX1.crt -t command
//+/req/#
A subscription to commands for a specific device can be done like this:
mosquitto_sub -d -h mqtt.bosch-iot-hub.com -p
8883
-u {auth-id}@{tenant-id} -P {secret} -k
30
--cafile ISRGRootX1.crt -t command
//{device-id}/req/#
Sending a response to a command
An authenticated gateway sends a device’s response to a command it has received on behalf of the device
Topic: command//${device-id}/res/${req-id}/${status-code}
req-id: the unique identifier of the command execution request
status-code: a HTTP status code indicating the outcome of executing the command.
Authentication: required with credentials of gateway
Payload:
(required) Arbitrary command payload
Send back command response:
mosquitto_pub -d -h mqtt.bosch-iot-hub.com -p
8883
-u {auth-id}@{tenant-id} -P {secret} --cafile ISRGRootX1.crt -t command
//{device-id}/res/{req-id}/200 -m '{"greetingDurationInSec": 23}'
Error Reporting via Error Topic
The default behaviour when an error occurs while publishing telemetry, event or command response messages is for the MQTT adapter in the device connectivity layer to close the MQTT connection to the device.
An alternative way of dealing with errors involves keeping the connection intact and letting the MQTT adapter publish a corresponding error message on a specific error topic to the device. To enable that behaviour, the device sends an MQTT SUBSCRIBE packet with a topic filter as described below on the same MQTT connection that is also used for publishing the telemetry, event or command response messages. Devices can subscribe with QoS 0 only. The adapter indicates the outcome of the subscription request by sending back a corresponding SUBACK packet. The SUBACK packet will contain Success - QoS 0 (0x00) for a valid error topic filter and will contain the Failure (0x80) value for an invalid or unsupported filter. In order to again activate the default error handling behaviour, the device can send an MQTT UNSUBSCRIBE packet to the adapter, including the same topic filter that has been used to subscribe.
The following sections define the topic filters to use for subscribing to error messages, and the resulting error message topic. Instead of the error topic path segment, the shorthand version e is also supported.
The following variables are used:
${endpoint-type}: The endpoint type of the device message that caused the error. Its value is either telemetry, event or the respective shorthand version. In case of a command response device message command-response or c-s is used.
${correlation-id}: The identifier that may be used to correlate the error message with the device message that caused the error. The identifier is either the value of a correlation-id property bag value contained in the device message topic, or the identifier is the packet-id of the device message if it was sent with QoS 1. Otherwise, a value of -1 is used.
${error-status}: The HTTP status code of the error that was caused by the device message.
Since the subscription on the error topic needs to be done on the same MQTT connection that is also used for publishing the telemetry, event or command response messages, the Mosquitto MQTT Command Line Client cannot be used. The MQTT CLI tool with its shell mode is an alternative that supports using one MQTT connection for both subscribing and publishing.
Receiving Error Messages
An authenticated device MUST use the following topic filter for subscribing to error messages:
error/[${tenant-id}]/[${device-id}]/#
Both the tenant and the device ID are optional. If specified, they MUST match the authenticated device’s tenant and/or device ID. Note that the authentication identifier used in the device’s credentials is not necessarily the same as the device ID.
The protocol adapter will publish error messages for the device to the following topic name
error/[${tenant-id}]/[${device-id}]/${endpoint-type}/${correlation-id}/${error-status}
The tenant-id and/or device-id will be included in the topic name if the tenant and/or device ID had been included in the topic filter used for subscribing to error messages.
Example
An example using the MQTT CLI that will produce an error output provided there is no downstream consumer for the device messages.
mqtt shell
con -V
3
-h [MQTT_ADAPTER_IP] -u [DEVICE]@[TENANT] -pw [PWD]
sub -t error
///# --qos 0 --outputToConsole
pub -t telemetry -m
'{"temp": 5}'
--qos
1
Using an explicit correlation id:
pub -t telemetry/?correlation-id=
123
-m
'{"temp": 5}'
--qos
1
Receiving Error Messages via a Gateway
A gateway MUST use one of the following topic filters for subscribing to error messages:
Topic Filter |
Description |
error//+/# |
Subscribe to error messages for all devices that the gateway is authorized to act on behalf of. |
error//${device-id}/# |
Subscribe to error messages for a specific device that the gateway is authorized to act on behalf of. |
The protocol adapter will publish error messages for the device to the following topic name
error/[${tenant-id}]/[${device-id}]/${endpoint-type}/${correlation-id}/${error-status}
The tenant-id and/or device-id will be included in the topic name if the tenant and/or device ID had been included in the topic filter used for subscribing to error messages.
Error Message Payload
The MQTT adapter publishes error messages with a UTF-8 encoded JSON payload containing the following fields:
Name |
Mandatory |
JSON Type |
Description |
code |
yes |
number |
The HTTP error status code. See the table below for possible values. |
message |
yes |
string |
The error detail message. |
timestamp |
yes |
string |
The date and time the error message was published by the MQTT adapter. The value is an ISO 8601 compliant combined date and time representation in extended format. |
correlation-id |
yes |
string |
The identifier that may be used to correlate the error message with the device message that caused the error. The identifier is either the value of a correlation-id property bag value contained in the device message topic, or the identifier is the packet-id of the device message if it was sent with QoS 1. Otherwise a value of -1 is used. |
The error message’s code field may contain the following HTTP status codes:
Code |
Description |
400 |
Bad Request, the request cannot be processed. A possible reason for this is an invalid PUBLISH topic. |
403 |
Forbidden, the device’s registration status cannot be asserted. |
404 |
Not Found, the device is disabled or does not exist. |
413 |
Request Entity Too Large, the request body exceeds the maximum supported size. |
429 |
Too Many Requests, the tenant’s message limit for the current period is exceeded. |
503 |
Service Unavailable, the request cannot be processed. Possible reasons for this include:
|
Example payload:
{
"code"
:
400
,
"message"
:
"malformed topic name"
,
"timestamp"
:
"2020-12-24T19:00:00+0100"
,
"correlation-id"
:
"5"
}
Error Handling
When a device publishes a telemetry, event or command response message and there is an error processing the message, the handling of the error depends on whether there is an error topic subscription for the device and whether an on-error property bag parameter was set on the topic used for sending the message.
If no error subscription is in place and no on-error parameter was set, the default error handling behaviour is to close the MQTT connection to the device. If the device has a subscription on the error topic (on the same MQTT connection the device uses for sending messages), the default behaviour is to keep the MQTT connection open unless a terminal error happens. The errors that are classified as terminal are listed below.
The adapter is disabled for the tenant that the client belongs to.
The authenticated device or gateway is disabled or not registered.
The tenant is disabled or does not exist.
When a terminal error occurs, the connection will always be closed irrespective of any on-error parameter or error subscription.
The following table lists the different behaviours based on the value of the on-error property bag parameter and the existence of an error subscription:
on-error topic parameter |
Error subscription exists |
Description |
default or value not set |
no |
The connection to the device will get closed (like with the disconnect option). |
disconnect |
no |
The connection to the device will get closed. |
ignore |
no |
The error will be ignored and a PUBACK for the message that caused the error will get sent. |
skip-ack |
no |
The error will be ignored and no PUBACK for the message that caused the error will get sent. |
default or value not set |
yes |
After having sent an error message on the error topic, the error will be ignored and a PUBACK for the message that caused the error will get sent (like with the ignore option). |
disconnect |
yes |
After having sent an error message on the error topic, the connection to the device will get closed. |
ignore |
yes |
After having sent an error message on the error topic, the error will be ignored and a PUBACK for the message that caused the error will get sent. |
skip-ack |
yes |
After having sent an error message on the error topic, the error will be ignored and no PUBACK for the message that caused the error will get sent. |
Example
A device wanting to have errors be always ignored can for example publish telemetry messages on this topic:
telemetry/?on-error=ignore
Troubleshooting
Issue |
Solution |
Device subscription for receiving commands was not acknowledged by the adapter. |
Verify that you are using QoS level 1 as QoS level 2 is not supported. Please note that some client programs might use QoS level 2 by default. If this is the case with your client software, please explicitly set QoS level to 1. |