Bosch IoT Device Management - will be discontinued by mid 2024

MQTT adapter

The MQTT protocol adapter exposes an MQTT topic hierarchy for publishing messages and events to the Telemetry and Event endpoints of the device connectivity layer.

The adapter does not behave like a general purpose MQTT broker.
Therefore the following applies to the adapter:

  • Supports MQTT 3.1.1 only.

  • Does not maintain session state for clients.
    Therefore setting the clean session flag on the client side does not have an impact on the adapter.
    The adapter will set the session present to 0 in the CONNACK packet in any case.

  • Ignores any Will contained in the CONNECT packet.

  • Only supports the topic names mentioned in the sections below.
    Custom topics are not supported.

  • Does not support retaining messages.
    However, if the retain flag is present in a message, then a x-opt-retain property is added to the downstream message and may be mapped in the header mappings of the connection (managed in the digital twin layer) .
    The application can then decide to react upon the presence of this annotation.


Table of contents

Establishing the connection

The MQTT adapter requires clients to authenticate during connection establishment. The adapter supports both username/password based authentication as well as client certificate based authentication. In order to do so, clients need to provide a username and a password in the MQTT CONNECT packet or a client certificate. The username must have the format auth-id@tenant-id.

The server certificate of the device connectivity layer is issued by Let’s Encrypt. Some devices directly trust the Let’s Encrypt root certificate, in such case downloading the server certificate is not necessary. If that is not the case, you need to import the Let’s Encrypt certificate of the device connectivity layer in your trust store to establish a TLS connection to the MQTT adapter.

For the examples below mosquitto_pub is used. mosquitto_pub needs the server certificate to establish a TLS connection to the device connectivity layer.

The device connectivity layer uses Let's Encrypt ISRG Root X1 certificate. The certificate can be downloaded as follows:

curl -o ISRGRootX1.crt https://letsencrypt.org/certs/isrgrootx1.pem


Inactivity timeout

The client should provide a Keep Alive value of less than 60s in the MQTT CONNECT packet to ensure that connection issues can be detected. Connected devices which are neither sending nor receiving data will be disconnected after one and a half times the Keep Alive time period, or after 60s, whichever comes first. To keep the connection open a device must send MQTT PINGREQ packages if needed to indicate to the server that the device is still active.

Note that if a device is subscribed to receive commands and the MQTT connection fails, a subsequent successful new connection and command subscription by the client may still result in commands not getting delivered until the Keep Alive time period has elapsed and the server detects the connection failure. This should be taken into consideration when choosing the Keep Alive value.

Communication patterns

Publishing telemetry data

The MQTT adapter supports publishing of telemetry data by means of MQTT PUBLISH packets using QoS 0 or QoS 1. The devices can optionally indicate the content type of the payload by setting the content-type property explicitly in the property-bag. The details regarding configuring the property-bag for telemetry messages can be found in Hono Documentation.

Publishing events

The MQTT adapter supports publishing of events by means of MQTT PUBLISH packets using QoS 1 only. The adapter will send an MQTT PUBACK packet to the client once the event has been stored in the messaging component of the device connectivity layer. The devices can optionally indicate a time-to-live duration for event messages and the content type of the payload by setting the hono-ttl and content-type properties explicitly in the property-bag. The details on configuring the property-bag for event messages can be found in Hono Documentation.

Command & control

The MQTT adapter supports receiving commands by means of MQTT SUBSCRIBE. Devices can subscribe to commands with QoS 0 or QoS 1. The responses to commands can being published using either QoS 0 or QoS 1.

All above mentioned communication patterns can be used by devices connecting directly to the MQTT protocol adapter or by gateways connecting to the MQTT protocol adapter and acting on behalf of the actual devices. See Gateway mode for a more detailed description.

Devices connected directly to MQTT adapter

Publishing telemetry data

  • Topic: telemetry

  • Authentication: required with credentials of device

  • Payload:

    • (required) Arbitrary payload

Example:

Upload a JSON string for device:

mosquitto_pub -h mqtt.bosch-iot-hub.com -p 8883 -u {auth-id}@{tenant-id} -P {password} -t telemetry -m '{"temp": 5}' --cafile ISRGRootX1.crt

Publishing events

  • Topic: event

  • Authentication: required with credentials of device

  • Payload:

    • (required) Arbitrary payload

Example:

Upload a JSON string for device:

mosquitto_pub -h mqtt.bosch-iot-hub.com -p 8883 -u {auth-id}@{tenant-id} -P {password} -t event -m '{"alarm": 1}' -q 1 --cafile ISRGRootX1.crt

Receiving commands

  • Topic Filter: command///req/#

  • The actual topic is structured as follows:

    • for one-way commands: command///req//{command-subject}

    • for request/response commands command///req/{req-id}/{command-subject}

    Parameters:

    command-subject: The command subject (e.g. the name of the command to execute)

  • QoS: Devices can subscribe with QoS 1 or QoS 0.

  • Authentication: required with credentials of device.

  • SUBACK response will contain "Failure" (0x80) value if subscription failed because of either an invalid topic filter or a (temporary) server error. SUBSCRIBE retries should be applied if needed.

  • The payload of the message contains the command payload.

Subscribe to command topic:

mosquitto_sub -d -h mqtt.bosch-iot-hub.com -p 8883 -u {auth-id}@{tenant-id} -P {secret} -k 30 --cafile ISRGRootX1.crt -t command///req/#

Sending a response to a command

  • Topic: command///res/{req-id}/{status-code}

    • req-id: the unique identifier of the command execution request

    • status-code: a HTTP status code indicating the outcome of executing the command.

  • Authentication: required with credentials of device

  • Payload:

    • (required) Arbitrary command payload

Send back command response:

mosquitto_pub -d -h mqtt.bosch-iot-hub.com -p 8883 -u {auth-id}@{tenant-id} -P {secret} --cafile ISRGRootX1.crt -t command///res/{req-id}/200 -m '{"greetingDurationInSec": 23}'

Gateways connected to MQTT adapter

Publishing telemetry data

  • Topic: telemetry/{tenant-id}/{device-id}

  • Authentication: required with credentials of gateway

  • Payload:

    • (required) Arbitrary payload


In Gateway Mode {device-id} is the ID of the actual device while credentials are the credentials of the gateway.

Example:

Upload a JSON string for device through gateway:

mosquitto_pub -h mqtt.bosch-iot-hub.com -p 8883 -u {auth-id}@{tenant-id} -P {password} -t telemetry/{tenant-id}/{device-id} -m '{"temp": 5}' --cafile ISRGRootX1.crt

Publishing events

  • Topic: event/{tenant-id}/{device-id}

  • Authentication: required with credentials of gateway

  • Payload:

    • (required) Arbitrary payload

In Gateway Mode {device-id} is the ID of the actual device while credentials are the credentials of the gateway.


Example:

Upload a JSON string for device through gateway:

mosquitto_pub -h mqtt.bosch-iot-hub.com -p 8883 -u {auth-id}@{tenant-id} -P {password} -t event/{tenant-id}/{device-id} -m '{"alarm": 5}' -q 1 --cafile ISRGRootX1.crt

Receiving commands

The gateway may receive commands for all devices in whose behalf it acts using topic filter command//+/req/# or for a specific device using topic filter command//${device-id}/req//{command-subject}.

  • Topic Filter to subscribe to commands for all devices in whose behalf a gateway acts: command//+/req/#

  • The actual topic is structured as follows:

    • for one-way commands for all devices: command//+/req//{command-subject}

    • for request/response commands for all devices: command//+/req/{req-id}/{command-subject}

  • Topic Filter to subscribe only to commands for a specific device, an authenticated gateway uses the topic filter command//${device-id}/req/#

  • The actual topic is structured as follows:

    • for one-way commands for a specific device: command//${device-id}/req//{command-subject}

    • for request/response commands for a specific device: command//${device-id}/req/{req-id}/{command-subject}

  • QoS: Devices can subscribe with QoS 1 or QoS 0.

  • Authentication: required with credentials of gateway.

  • SUBACK response will contain "Failure" (0x80) value if subscription failed because of either an invalid topic filter or a (temporary) server error. SUBSCRIBE retries should be applied if needed.

  • The payload of the message contains the command payload.

​Subscribe to command topic:

​A subscription to commands for all devices that a gateway acts on behalf of looks like this:

mosquitto_sub -d -h mqtt.bosch-iot-hub.com -p 8883 -u {auth-id}@{tenant-id} -P {secret} -k 30 --cafile ISRGRootX1.crt -t command//+/req/#

A subscription to commands for a specific device can be done like this:

mosquitto_sub -d -h mqtt.bosch-iot-hub.com -p 8883 -u {auth-id}@{tenant-id} -P {secret} -k 30 --cafile ISRGRootX1.crt -t command//{device-id}/req/#

Sending a response to a command

An authenticated gateway sends a device’s response to a command it has received on behalf of the device

  • Topic: command//${device-id}/res/${req-id}/${status-code}

    • req-id: the unique identifier of the command execution request

    • status-code: a HTTP status code indicating the outcome of executing the command.

  • Authentication: required with credentials of gateway

  • Payload:

    • (required) Arbitrary command payload

Send back command response:

mosquitto_pub -d -h mqtt.bosch-iot-hub.com -p 8883 -u {auth-id}@{tenant-id} -P {secret} --cafile ISRGRootX1.crt -t command//{device-id}/res/{req-id}/200 -m '{"greetingDurationInSec": 23}'

Error Reporting via Error Topic

The default behaviour when an error occurs while publishing telemetry, event or command response messages is for the MQTT adapter in the device connectivity layer to close the MQTT connection to the device.

An alternative way of dealing with errors involves keeping the connection intact and letting the MQTT adapter publish a corresponding error message on a specific error topic to the device. To enable that behaviour, the device sends an MQTT SUBSCRIBE packet with a topic filter as described below on the same MQTT connection that is also used for publishing the telemetry, event or command response messages. Devices can subscribe with QoS 0 only. The adapter indicates the outcome of the subscription request by sending back a corresponding SUBACK packet. The SUBACK packet will contain Success - QoS 0 (0x00) for a valid error topic filter and will contain the Failure (0x80) value for an invalid or unsupported filter. In order to again activate the default error handling behaviour, the device can send an MQTT UNSUBSCRIBE packet to the adapter, including the same topic filter that has been used to subscribe.

The following sections define the topic filters to use for subscribing to error messages, and the resulting error message topic. Instead of the error topic path segment, the shorthand version e is also supported.

The following variables are used:

  • ${endpoint-type}: The endpoint type of the device message that caused the error. Its value is either telemetry, event or the respective shorthand version. In case of a command response device message command-response or c-s is used.

  • ${correlation-id}: The identifier that may be used to correlate the error message with the device message that caused the error. The identifier is either the value of a correlation-id property bag value contained in the device message topic, or the identifier is the packet-id of the device message if it was sent with QoS 1. Otherwise, a value of -1 is used.

  • ${error-status}: The HTTP status code of the error that was caused by the device message.


Since the subscription on the error topic needs to be done on the same MQTT connection that is also used for publishing the telemetry, event or command response messages, the Mosquitto MQTT Command Line Client cannot be used. The MQTT CLI tool with its shell mode is an alternative that supports using one MQTT connection for both subscribing and publishing.

Receiving Error Messages

An authenticated device MUST use the following topic filter for subscribing to error messages:

error/[${tenant-id}]/[${device-id}]/#

Both the tenant and the device ID are optional. If specified, they MUST match the authenticated device’s tenant and/or device ID. Note that the authentication identifier used in the device’s credentials is not necessarily the same as the device ID.

The protocol adapter will publish error messages for the device to the following topic name

error/[${tenant-id}]/[${device-id}]/${endpoint-type}/${correlation-id}/${error-status}

The tenant-id and/or device-id will be included in the topic name if the tenant and/or device ID had been included in the topic filter used for subscribing to error messages.

Example

An example using the MQTT CLI that will produce an error output provided there is no downstream consumer for the device messages.

mqtt shell
con -V 3 -h [MQTT_ADAPTER_IP] -u [DEVICE]@[TENANT] -pw [PWD]
sub -t error///# --qos 0 --outputToConsole
pub -t telemetry -m '{"temp": 5}' --qos 1
 
Using an explicit correlation id:
 
pub -t telemetry/?correlation-id=123 -m '{"temp": 5}' --qos 1

Receiving Error Messages via a Gateway

A gateway MUST use one of the following topic filters for subscribing to error messages:

Topic Filter

Description

error//+/#
error/${tenant-id}/+/#

Subscribe to error messages for all devices that the gateway is authorized to act on behalf of.

error//${device-id}/#
error/${tenant-id}/${device-id}/#

Subscribe to error messages for a specific device that the gateway is authorized to act on behalf of.

The protocol adapter will publish error messages for the device to the following topic name

error/[${tenant-id}]/[${device-id}]/${endpoint-type}/${correlation-id}/${error-status}

The tenant-id and/or device-id will be included in the topic name if the tenant and/or device ID had been included in the topic filter used for subscribing to error messages.

Error Message Payload

The MQTT adapter publishes error messages with a UTF-8 encoded JSON payload containing the following fields:

Name

Mandatory

JSON Type

Description

code

yes

number

The HTTP error status code. See the table below for possible values.

message

yes

string

The error detail message.

timestamp

yes

string

The date and time the error message was published by the MQTT adapter. The value is an ISO 8601 compliant combined date and time representation in extended format.

correlation-id

yes

string

The identifier that may be used to correlate the error message with the device message that caused the error. The identifier is either the value of a correlation-id property bag value contained in the device message topic, or the identifier is the packet-id of the device message if it was sent with QoS 1. Otherwise a value of -1 is used.

The error message’s code field may contain the following HTTP status codes:

Code

Description

400

Bad Request, the request cannot be processed. A possible reason for this is an invalid PUBLISH topic.

403

Forbidden, the device’s registration status cannot be asserted.

404

Not Found, the device is disabled or does not exist.

413

Request Entity Too Large, the request body exceeds the maximum supported size.

429

Too Many Requests, the tenant’s message limit for the current period is exceeded.

503

Service Unavailable, the request cannot be processed. Possible reasons for this include:

  • There is no consumer of telemetry data for the given tenant connected to the device connectivity layer, or the consumer has not indicated that it may receive further messages (not giving credits).

  • If the QoS level header is set to 1 (at least once semantics), the reason may be:

    • The consumer has indicated that it didn’t process the telemetry data.

    • The consumer failed to indicate in time whether it has processed the telemetry data.

Example payload:

{
"code": 400,
"message": "malformed topic name",
"timestamp": "2020-12-24T19:00:00+0100",
"correlation-id": "5"
}

Error Handling

When a device publishes a telemetry, event or command response message and there is an error processing the message, the handling of the error depends on whether there is an error topic subscription for the device and whether an on-error property bag parameter was set on the topic used for sending the message.

If no error subscription is in place and no on-error parameter was set, the default error handling behaviour is to close the MQTT connection to the device. If the device has a subscription on the error topic (on the same MQTT connection the device uses for sending messages), the default behaviour is to keep the MQTT connection open unless a terminal error happens. The errors that are classified as terminal are listed below.

  • The adapter is disabled for the tenant that the client belongs to.

  • The authenticated device or gateway is disabled or not registered.

  • The tenant is disabled or does not exist.


When a terminal error occurs, the connection will always be closed irrespective of any on-error parameter or error subscription.


The following table lists the different behaviours based on the value of the on-error property bag parameter and the existence of an error subscription:

on-error topic parameter

Error subscription exists

Description

default or value not set

no

The connection to the device will get closed (like with the disconnect option).

disconnect

no

The connection to the device will get closed.

ignore

no

The error will be ignored and a PUBACK for the message that caused the error will get sent.

skip-ack

no

The error will be ignored and no PUBACK for the message that caused the error will get sent.

default or value not set

yes

After having sent an error message on the error topic, the error will be ignored and a PUBACK for the message that caused the error will get sent (like with the ignore option).

disconnect

yes

After having sent an error message on the error topic, the connection to the device will get closed.

ignore

yes

After having sent an error message on the error topic, the error will be ignored and a PUBACK for the message that caused the error will get sent.

skip-ack

yes

After having sent an error message on the error topic, the error will be ignored and no PUBACK for the message that caused the error will get sent.

Example

A device wanting to have errors be always ignored can for example publish telemetry messages on this topic:

telemetry/?on-error=ignore

Troubleshooting

Issue

Solution

Device subscription for receiving commands was not acknowledged by the adapter.

Verify that you are using QoS level 1 as QoS level 2 is not supported. Please note that some client programs might use QoS level 2 by default. If this is the case with your client software, please explicitly set QoS level to 1.