IoT end-to-end encryption with E4 (2/n): architecture and integration

The previous post in the series introduced the E4 protocol, which brings end-to-end encryption to IoT networks in a scalable way. We now describe how simple and unobtrusive E4 is to integrate in an existing infrastructure. We’ll start with a basic example of an IoT architecture, present the limitations in terms of data security, and then introduce E4 components and their integration.

An typical IoT architecture

A common industrial IoT architecture includes a multitude of connected devices, reporting their data such as measurements to a processing back-end. In the example depicted below, the MQTT transport protocol is used, such that devices and the back-end connect to an MQTT message broker in order to exchange messages according to a publish/subscribe logic.

Because devices directly publish their data as cleartext messages, this data is exposed to the message broker, which can read its content, modify it, and create fake messages on behalf of the devices. This can happen in different scenario, for example when:

  • The broker is operated by a third party, such as a cloud operator
  • The broker is compromised by an attacker, who then controls its operation
  • The broker database is compromised, and therefore all its messages

Other network components (routers, gateways, etc.) can also do the same if the connection between devices and the broker is not protected.

In other words important security properties are missing in this architecture:

  • Data authenticity and integrity of messages
  • Data confidentiality

The same concerns apply when data is sent to the devices, such as firmware or configuration updates, as well as when devices communicate with each other.

E4 integration

The E4 solution aims to protect IoT data without making any change to the message broker (or similar component), and in a way that it mostly independent of the transport protocol (in this example, MQTT). As you can see in the image below, our server component (C2) connects to the message broker like any other device, and other network components (devices and back-end) integrate our software library in order to support the E4 security layer. The subsequent sections explain how this integration is done.


We defines a few terms:

  • E4 clients (or just "clients"): In E4, clients are anything able to publish or receive messages. In our examples, the devices, sensors or processing backend are all identified as clients. E4 clients must each have a unique identifier, and a secret key.
  • C2 server (or just "C2"): The back-end component of E4, which is seen by the network as just another device. C2 sends control messages to E4 clients and performs security monitoring operations, among others.
  • Topics: MQTT topics are the message queues where messages will be published and read from. In E4, topics can be associated with a secret key, which clients will use to protect messages tagged with this topic.
  • Protect/Unprotect: Instead of just encryption/decryption, E4’s security layer can also provide integrity, authenticity, and replay protection. That’s why we refer to the message transformation as "protection" rather than only encryption.

Modifications to clients

To integrate E4, the main modifications take place at the applications level, which need to send or receive messages on the clients. They need to integrate the E4 library in order to:

  • Protect a message before publishing it. This ensure that intercepted messages can’t be read nor modified by unauthorized parties.
  • Unprotect a message after receiving it. This allow to retrieve the original message payload from a protected message.
  • Subscribe to the device’s control topic. This allow the C2 server to send control messages to this unique client.
  • Store an identifier and related key material.
  • Process control messages sent by the C2.

C2 integration

The C2 is E4’s key management server, whose main roles are to:

  • Keep track of client devices and their associated cryptographic keys.
  • Identify communication topics, and their associated cryptographic keys.
  • Send control messages to the clients.
  • Perform analytics to monitor the network.

The C2 exposes an API (gRPC or HTTP) allowing to register all your clients and topics, and securely provision keys to devices in a manual or automated way. To do so, it only needs to reach the MQTT broker, as a normal client (ideally with MQTT QoS parameter set to 2).

Besides its core service, C2 provides additional and optional services:

  • A simple web-based UI, to manage clients and topics manually. See our public demo
  • A command-line tool (c2cli), allowing scripting and batch processing for your clients and topics management.
  • An automation engine to define, manage and automate key rotation policies (which will be presented in a future post in this series).

As depicted above, additional services directly connect to the core C2 service, meaning there is still no change to make to your existing infrastructure.

C2 database

The C2 can use popular SQL RDBMSs (such as MySQL, PostgreSQL, SQLite) to store your devices and topics. All the cryptographic keys are stored encrypted, and only decrypted when needed, that is, when transmitting them to clients in control messages.

In its simplest setting, database records are encrypted using a key controlled by the C2 service. Depending on the security needs and available technology, the following components can be used to manage the database encryption key:

  • HSM via PKCS#11 interface
  • Dedicated key management service such as Hashicorp’s Vault
  • Cloud-based key management services such as AWS KMS

Control messages

In order to control each devices internal state, such as their identity key (used by the C2 server to encrypt the device’s control messages), or the list of topic keys (used to encrypt or decrypt the data transmitted or received by the client), the C2 service emit control messages, which can be of one of the following types:

  • SetTopicKey to set a key for a given topic, allowing the device to read or publish messages on this topic.
  • RemoveTopic to remove a topic key from the client, preventing the device to read or publish messages on the topic.
  • ResetTopics to remove all the topic keys stored by th client.
  • SetIDKey to set the device identity key, required to process control messages.

As we’ll describe in a future post, E4 support symmetric-key and public-key cryptography modes. In the public-key mode (which is the safest), the following messages are used:

  • SetPubKey to add the public key of another device in the local storage, allowing to authenticate the message sender.
  • RemovePubKey to remove a stored public key from the client’s storage.
  • ResetPubKeys to remove all the public keys stored by the client.

Control messages are protected using the current device key, and published by the C2 on the device’s control topic (created from the client unique identifier we mentioned above).

The processing of control messages is implemented in the client software, as you can see for example in E4’s Go client.


E4 allows you to protect your IoT data with minimal changes required, and a short integration and testing period. E4 does not require any change to the message broker, nor additional hardware or software capabilities on the clients.

E4 is also robust and resilient to outages: although the C2 services are designed for scalability and high-availability, devices can still communicate reliably if these are unavailable or are temporarily down for maintenance..

If you’re interested in learning more or have any question, feel free to contact us!