Lightweight crypto standards (sorry NIST)

NIST’s running a Lightweight Cryptography project in order to standardize symmetric encryption primitives that are lighter than the established NIST standards (read: AES and its modes of operations). NIST claims that “[because] the majority of current cryptographic algorithms were designed for desktop/server environments, many of these algorithms do not fit into constrained devices.” This is good motivation for the competition, however it’s factually incorrect: AES today fits in almost all IoT-ish chips and has even been used for bus and memory encryption.

But that’s not the point of this post—for more on that subject, come hear our talk at NIST’s Lightweight Cryptography Workshop in about 10 days, a talk derived from one of our previous posts, and based on a multitude of real examples. (See also this recent Twitter thread.)

Sorry NIST, this post is not about you, but about another standardization body: ISO, and specifically its ISO/IEC 29192 class of lightweight standards within the 35.030 category (“IT Security – including encryption”). This category includes no less than 9 standards relating to lightweight cryptography, which are (copying verbatim from the ISO page, links included):

ISO/IEC 29192-1:2012Information technology — Security techniques — Lightweight cryptography — Part 1: General90.93ISO/IEC JTC 1/SC 27
ISO/IEC 29192-2IT security techniques — Lightweight cryptography — Part 2: Block ciphers60.00ISO/IEC JTC 1/SC 27
ISO/IEC 29192-2:2012Information technology — Security techniques — Lightweight cryptography — Part 2: Block ciphers90.92ISO/IEC JTC 1/SC 27
ISO/IEC 29192-3:2012Information technology — Security techniques — Lightweight cryptography — Part 3: Stream ciphers90.93ISO/IEC JTC 1/SC 27
ISO/IEC 29192-4:2013Information technology — Security techniques — Lightweight cryptography — Part 4: Mechanisms using asymmetric techniques90.93ISO/IEC JTC 1/SC 27
ISO/IEC 29192-4:2013/AMD 1:2016Information technology — Security techniques — Lightweight cryptography — Part 4: Mechanisms using asymmetric techniques — Amendment 160.60ISO/IEC JTC 1/SC 27
ISO/IEC 29192-5:2016Information technology — Security techniques — Lightweight cryptography — Part 5: Hash-functions60.60ISO/IEC JTC 1/SC 27
ISO/IEC 29192-6:2019Information technology — Lightweight cryptography — Part 6: Message authentication codes (MACs)60.60ISO/IEC JTC 1/SC 27
ISO/IEC 29192-7:2019Information security — Lightweight cryptography — Part 7: Broadcast authentication protocols60.60ISO/IEC JTC 1/SC 27

Granted, few people and industries care about ISO/IEC standards, and you probably don’t, neither do we really to be honest—the fee around $100 to access the ISO standard documents probably doesn’t help in making ISO algorithms more popular.

We nonetheless think that it would be worthwhile to have a list of lightweight symmetric algorithms (whatever your weight metric) that received the blessing of allegedly some competent cryptographers, and therefore that are presumably safe to use. We’ve therefore reviewed what ISO has to offer so that you don’t have to, and summarize the fruit of our research in the remainder of this post:

Block ciphers

ISO/IEC 29192-2:2012 (thus, from 2012) standardizes these two block ciphers (literally just the block ciphers, and not any mode, which are covered in another ISO standard [nothing surprising here: ECB, CBC, OFB, CFB, CTR]):

  • PRESENT, anagram and little sister of SERPENT, designed in 2007, and the least Google-friendly cipher ever. Then marketed as ultra-lightweight, PRESENT is a simple substitution-permutation network with 4-bit S-boxes, which is more hardware-friendly than it is software-friendly, but will perform well enough almost everywhere. It has 64-bit blocks and a key of 80 or 128 bits.
  • CLEFIA, also designed in 2007, from Sony, initially aimed for DRM applications, is a boring Feistel scheme with 128-bit blocks and a key of 128, 192, or 256 bits. CLEFIA isn’t really more lightweight than AES, and I guess Sony lobbied for its standardization.

Stream ciphers

ISO/IEC 29192-3:2012 standardizes these two stream ciphers (again, one bit-oriented and one byte-oriented):

  • Trivium, one of the winners of the eSTREAM project, the favorite cipher of the DEF CON conference, is a minimalistic, bit-oriented stream cipher that is a simple combination of shift registers. Trivium is arguably the lightweightest algorithm in this post.
  • Enocoro, whose existence I had completely forgotten until writing this post. Designed by the Japanese firm Hitachi, Enocoro was submitted to the CRYPTREC contest in 2010 (see this cryptanalysis evaluation), yet ended up not being selected. Enocoro is a byte-oriented feedback shift register with a conservative design. It supports 80- and 128-bit keys, and is most probably safe (and safer with 128-bit keys).

Hash functions

This time not two but three algorithms, as defined in ISO/IEC 29192-5:2016:

  • PHOTON, designed in 2011, combines the simplicity of the sponge construction and the security guarantees of the AES permutation (no, Joan Daemen is not a designer of PHOTON). The permutation is optimized for low-end platforms, so you can’t directly reuse AES code/silicon. PHOTON comes in multiple versions, depending on the security level you want (the higher the security, the bigger the state).
  • SPONGENT, also from 2011, is to PRESENT what PHOTON is to AES. Nuff said.
  • Lesamnta-LW is different. It’s not a sponge but has a SHA-like round function and targets 120-bit security with a single version. It’s also less light than the above two.

MACs

Of the three algorithms listed in ISO/IEC 29192-6:2019, I was only familiar with one Chaskey, which is essentially a 32-bit version of SipHash. The other two are

  • LightMAC, which is actually a MAC construction rather than strictly speaking an algorithm. And since I didn’t pay the 118 CHF to read the full document, I don’t really know what ISO standardized here (the mode itself? an instantiation with a specific block cipher? Help please.)
  • “Tsudik’s keymode”, apparently from this 1992 paper by Gene Tsudik, which discussed the secret-prefix and secret-suffix MACs and proposes the hybrid version, which may or may not be what this “keymode” is about. I’ve no idea why it ended up being standardized in 2019.

AEAD

Nothing here, ISO hasn’t standardized authenticated ciphers.

Summary

All these algorithms are okay, but their being ISO standards doesn’t say much and doesn’t mean that they’re necessarily superior to others, just that some people bothered submitting them and lobbying for their standardization.

How to lock a GitHub user out of their repos (bug or feature?)

We accidentally discovered this while working with our friends from X41, when GitHub denied me access to my private repositories; for example git pull of git clone would fail and write “ERROR: Permission to <account>/<repo>.git denied to deploy key”. I asked for help but nobody found the root cause.

Here’s the problem: GitHub recently introduced deploy keys, or keys granting read-only right to a server to best automate deployment scripts. This is a great feature, however it can be abused as follows to lock a GitHub user out of their repositories:

  1. Find one or more SSH public keys belonging to the victim, but not associated to their GitHub account. For example, you may happen to know SSH keys associated to another project, or you may use public keys from https://gitlab.com/username.keys (comparing with https://github.com/username.keys).
  2. Add these SSH public keys as deploy keys of one of your GitHub projects (in Settings > Deploy keys > Add deploy key). Let’s say this project is at github.com/attacker/repo.
  3. When connecting to GitHub over SSH, the victim will then be identified at attacker/repo (for example when doing ssh -T [email protected]), if and only if at least of the keys added as deploy keys is prioritized by SSH over any key linked to the GitHub account.

For example, if you have private key files id_ecdsa_github and id_ecdsa_gitlab in your ~/.ssh/ directory, and if SSH offers the public key id_ecdsa_gitlab.pub first, and if the attacker has added that key as deploy key, then GitHub will identify you as attacker/repo when you’ll try to connect to one of your repositories, thereby denying you access to your private repositories (and only granting read access to public ones).

The way SSH prioritizes key can be found in its code, and also varies depending on whether you use ssh-agent (typically, when caching passphrase-protected private keys).

This behavior of GitHub and SSH can therefore be exploited to lock GitHub users out of their private repositories (via command-line SSH), and arguably qualifies as a denial-of-service attack. Note that it doesn’t require any action from the victim, and only needs public information (if we assume that public keys are public). We’ve lost one afternoon of work investigating the issue, being involuntarily attacked by X41. After understanding what happened, we asked them to remove our (freshly generated) key from their repo’s deploy keys, which indeed solved the issue.

You can argue that it’s the responsibility of users to properly configure their ~/.ssh/config, which of course avoids the attack, and that that behavior is acceptable. This is presumably the opinion of GitHub, since they responded to our bug bounty entry that “it does not present a security risk”. GitHub probably has good reasons to ignore the risk, but from our perspective the potential annoyance and DoS risk to GitHub users is not be negligible.

The problem can probably be fixed, although it’s not be straightforward, since SSH authentication would have to depend not only on the host name, but also on the repository accessed (whereas access control to a repository is typically enforced after SSH authentication).

Update: As @stuartpb noticed, this behavior can also be exploited by adding a victim’s public keys to a user account (new or repo-less), and is not specific to deploy keys.

Cryptography in industrial embedded systems: our experience of the needs and constraints

A common model is that industrial embedded systems (a.k.a. IoT, M2M, etc.) need small, fast, low-energy crypto primitives—requirements often summarized by the “lightweight” qualificative. For example, a premise of NIST’s Lightweight Cryptography standardization project is that AES is not lightweight enough, and more generally that “the majority of current cryptographic algorithms were designed for desktop/server environments, many of these algorithms do not fit into constrained devices”—note the implicit emphasis on size.

In this article we share some observations from our experience working with industrial embedded systems in various industries, on various platforms and using various network protocols. We notably challenge the truism that small devices need small crypto, and argue that finding a suitable primitive is usually the simplest task that engineers face when integrating cryptography in their products. This article reviews some of the other problems one has to deal with when deploying cryptography mechanism on “lightweight” platforms.

We’re of course fatally subject to selection bias, and don’t claim that our perspective should be taken as a reference or authoritative. We nonetheless hope to contribute to a better understanding of what are the “constrained devices” that NIST refers to, and more generally of “real-world” cryptography in the context of embedded systems.

Few details can be share, alas, about our experience. Let us only say that we’ve designed, integrated, implemented, or reviewed cryptographic components in systems used in automotive, banking, content protection, satellite communications, law enforcement technology, supply chain management, device tracking, or healthcare.

AES is lightweight enough

Most of the time. Ten years ago we worked on a low-cost RFID product that had to use something else that AES, partially for performance reasons. Today many RFID products include an AES engine, as specified for example in the norm ISO/IEC 29167-10.

Systems on chips and boards for industrial application often include AES hardware, and when they don’t a software implementation of AES comes at acceptable costs. For example, chips from the very popular STM32 family generally provide readily available AES in different modes of operations.

An example perhaps of a very low-cost device that uses a non-AES primitive would be the Multos Step/One, which is EMVCo-compliant. In this case, 3DES was chosen instead. ATM Encrypting-PIN-Pads frequently use 3DES. We believe that the continued use of 3DES has more to do with compatibility and cost of replacement than the prohibitive cost of running a wider block cipher.

Of course an algorithm more compact and faster than AES wouldn’t hurt, but the benefits would have to justify the integration costs.

Choosing primitives is a luxury

More than once we faced the following challenge: create a data protection mechanism given the cryptography primitives available on the devices, which could for example be AES-GCM and SHA-256 only. We may have to use AES-GCM and SHA-256 because they’re standards, because they’re efficiently implemented (for example through hardware accelerators), or for the sake of interoperability. Note that a platform may give you access to AES-GCM, but not to the AES core directly, so you can’t use it to implement (say) AES-SIV.

If you want to use another algorithm than the ones available, you have to justify that it’s worth the cost of implementing, integrating and testing the new primitive. AES-GCM is not perfect (risk of nonce reuse, etc.), but the risk it creates is usually negligible compared to other risks. The situation may be different with AES-ECB.

Statelessness

Not all platforms are stateful, or reliably stateful. This means that you can’t always persistently store a counter, seed, or other context-dependent on the device. The software/firmware on the platform may be updatable over-the-air, but not always. The keys stored on the device may not be modifiable after the personalization phase of the production cycle.

Randomness

The platform may not offer you a reliable pseudorandom generator, or it may only have some low-entropy non-cryptographic generator, anyway that can be a severe limitation, especially if you realize this after proudly completing an implementation of ECDSA.

There are well-known workarounds of course, such as deterministic ECDSA and EdDSA for ECC signatures, or AES-SIV for authenticated encryption (this robustness to weak/non-randomness is the main reason why we chose to make it the default cipher in our company’s product). But sometimes it can get trickier, when you really need some kind of randomness yet can’t fully trust the PRNG (it’s more fun when the platform is stateless).

It’s not only about (authenticated) encryption

When no established standard such as TLS is used—and sometimes even when it is—the security layer is typically implemented at the application layer between the transport and business logic. (Authenticated) encryption is a typical requirement, but seldom the only one: you may have to worry about replay attacks or have to “obfuscate” some metadata or header information, for example to protect anonymity.

In an ideal world, you ought to use a thoroughly-designed, provably-secure, peer-reviewed protocol. But 1) such a thing likely doesn’t exist, and 2) even when it does it would probably not be suitable to your use case, for example if the said protocol requires a trusted third party or three network round-trips.

Message size limitations

True story: “Our clear payload is N bytes; the protected payload must be N bytes too, and must be encrypted and authenticated.” That’s when you have to be creative. Such a situation can occur with protocols such as Bluetooth Low-Energy or WAN protocols such as LoRaWAN or Sigfox (where the uplink payloads are 12 bytes and downlink payloads 8 bytes).

Even when you can afford some overhead to send a nonce and a tag, this may come at prohibitive cost if we’re talking of millions of messages and a per-volume pricing model. In other contexts, additional payload size can increase the risk of packet loss.

Network unreliability

It’s not just about TCP being reliable (guaranteed, in-order packet delivery) and UDP being not. Protocols running on top of TCP can have their own reliability properties caused by the way they transmit messages. For example, MQTT (when running over TCP) guarantees message delivery, but not in-order.

Whatever protocol is used, devices may have no way to transmit nor receive messages for a certain period of time. For example, communicating with satellites in non-geostationary orbit, or devices that are out of range for periods of time, such as aircraft, ships or smart meters as a measuring driver passes by.

An excellent engineering question is to ask how one would reliably transmit data from Mars, particularly where that data should be processed as quickly as possible on receipt. If not Mars, then further away. At such great distances, what is instantaneous for us starts to take seconds or minutes of time. The answer is to resend on a broadcast channel repeatedly, usually as an illustrative example of UDP/broadcast protocols. In these cases, packets are expected to be lost and round-trips are impossible—there is no way to confirm receipt.

Such limitations often prevent the use of crypto protocols adding RTTs, or requiring even a moderate level of synchronization with other devices. Unreliable network becomes particularly fun when implementing key rotation or key distribution mechanisms.

When crypto is too big

Crypto can be too big (in code size, or RAM usage) for certain platforms; the main cases we’ve encountered are when public-key operations are impossible, or when a TLS implementation takes too much resources. Although there are good TLS implementations for constrained platforms (such as ARM’s mbedTLS, wolfSSL, or BearSSL), they may include a lot of code to support the TLS standards and operations such as parsing certificates. Even the size of a TLS-PSK stack can prohibitive—and not because of AES.

Sometimes public-key cryptography is even possible within the limited capacity of the device, but the limiting factor is the protocol. An example of such a problem can be found in BearSSL’s documentation. Quoting Thomas Pornin’s TLS 1.3 Status, when streaming ASN.1 certificates, the usual order is end-entity first, CAs later. In any given object, the public key follows the certificate. EdDSA combines the public key and data when signing. Thus, in order to validate a signature, the entire certificate must be buffered until the public key can be extracted.

Now while a highly constrained device may simply use PSK and avoid this problem entirely, it is also true that the device may be capable of Ed25519 signatures even without sufficient RAM to buffer large certificates. This problem arises entirely from the choice of PureEdDSA rather than HashEdDSA in TLS 1.3.

Untrusted infrastructure

More often than you might expect, the infrastructure we use should not be trusted. In the MQTT context, this means brokers. In other context this means wireless repeaters, conversion gateways between protocols such as communication via SMS and so on. In the context of currently proposed IoT standards, these nodes are often assumed trusted and capable of re-encrypting for each hop using TLS where possible, or some other point-to-point protocol.

We believe that the implicit trust in the infrastructure by having it handle keys invites a far greater risk than the challenges of underpowered devices.

Conclusions

Cryptography on constrained devices can pose many problems, but the speed and size of symmetric primitives (ciphers, hash functions) is rarely one, at least in our experience (YMMV).

We can’t ignore that economics and risk management play into cryptography. Standard NIST cryptography primitives and NSA Suites A & B, for example, were designed to provide the US Government with an assurance that data is protected for the lifetime of the relevant classified information—on the order of magnitude of 50 years. It took time for the community to gain confidence in AES, but it’s now widely and globally trusted—anyway, safe block ciphers are easy to design; even DES nor GOST have never really been broken.

Lightweight cryptography might be suitable where such expectations of long-term security do not hold, and would allow the use of a very “lightweight” component. An extreme example is that or memory encryption, or “scrambling”, where only a handful of high-frequency cycles can be allocated.

The open question is whether we can design algorithms to match this requirement, bearing in mind that we have no ability to predict future developments. Looking back at history, requirements are driven by applications on which the public research community has little view. As highlighted in this article, said requirements often involve various components and technologies, which make the engineering problem difficult to approach to outsiders.

E4 vs. (D)TLS #IoTencryption

Today at Teserakt we discussed the benefits of using our end-to-end encryption protocol E4 instead of TLS with pre-shared keys (PSK), as sometimes used on low-end devices that don’t use public-key cryptography. The discussion started after a call with a start-up that is specialized in low-power WAN networks and uses DTLS with PSK for protecting data sent to and received from devices. We thought we would write this quick post to share our thoughts and encourage readers to share their experience with similar protocols.

The following points (in arbitrary order) are the main benefits of E4 that we identified, and in our experience cover some of the most important problems encountered when attempting to deploy encryption on low-power devices:

  • An application layer protocol and can therefore go over other protocols, properly end-to-end, whereas (D)TLS is not end-to-end however you cut it.
  • “No ridiculously complex standards (TLS requires full ASN.1 parser)”
  • Remains secure if the device has neither a PRNG nor a clock.
  • Simpler key management (no PKI, X.509, etc.).
  • 0-RTT; E4 doesn’t need to perform a handshake mechanism to start sending encrypted data.
  • Much smaller code and RAM footprint. In particular no need to allocate MBs of memory to process certificate chains unlike TLS.

A last remark: as an alternative to (D)TLS for low-end platforms we’ll be evaluating the Noise family of protocols and in particular Rust implementations optimized for ARM-based chips. More on this in a future post 🙂

A secure element host library bug

As part of our mission to bring state of the art security to embedded devices, Teserakt occasionally works with customers to review the security of their products. With our cryptographic expertise we are often asked to look at the design of cryptographic solutions, but we also review code for potential security-sensitive issues.

One of our customers had chosen to use the A71CH Secure Element, which at its simplest is a smartcard element for embedded circuitry capable of public key cryptography on behalf of the host MCU. Such devices, like smartcards, store public-private keypairs in such a way that the private component is next-to-impossible to extract (in theory), as such, for high-assurance systems they are hugely important.

The host MCU, the device to which the secure element is connected, speaks to the A71CH device using the GlobalPlatformPro smartcard standards, specifically SCP11. To facilitate this, NXP provide a host library that can be linked with embedded firmware to talk to the device.

During our review, we were asked to look specifically at components of the NXP library code to ensure this was also free of potential errors. Here we found some classic overflows. First, the deprecated code for selecting an applet is as follows:

U32 GP_SelectApplet(U8 * pAppletName, U8 appletNameLength, U8 * pResponse, U32 * pResponseLength)
{
U32 st = 0;
U8 txBuf[128];
U8 len = appletNameLength;
txBuf[0] = CLA_ISO7816;
txBuf[1] = INS_GP_SELECT;
txBuf[2] = 0x04; txBuf[3] = 0x00;
txBuf[4] = len;
// AV: buffer overflow here. max(U8)=255, sizeof txBuf=128.
// fix this with:
// #DEFINE TRANS_APPLET_OFFSET 5
// memcpy(&txBuf[TRANS_APPLET_OFFSET], pAppletName,min(appletNameLength, sizeof(txBuf) - TRANS_APPLET_OFFSET));
memcpy(&txBuf[5], pAppletName, len);
txBuf[5+ len] = 0x00;
assert(pAppletName != NULL);
st = smCom_TransceiveRaw(txBuf, 6+len, pResponse, pResponseLength);
return st;
}

We have included our review comments. The problem should be obvious to anyone familiar with secure coding in C: the pAppletName buffer can contain more than 128 bytes of data, which will overflow into the return address and onto the stack.

Having seen this, we next looked at the non-deprecated code for applet selection. This starts at:

U16 GP_Select(U8 *appletName, U16 appletNameLen, U8 *responseData, U16 *responseDataLen)
{
U16 rv = 0;
apdu_t apdu;
apdu_t * pApdu = (apdu_t *) &apdu;
U8 isOk = 0x00;
assert(appletName != NULL);
assert(responseData != NULL);
pApdu->cla = CLA_ISO7816;
pApdu->ins = INS_GP_SELECT;
pApdu->p1 = 0x04;
pApdu->p2 = 0x00;
AllocateAPDUBuffer(pApdu);
SetApduHeader(pApdu, USE_STANDARD_APDU_LEN);
smApduAppendCmdData(pApdu, appletName, appletNameLen);
rv = (U16)scp_Transceive(pApdu, SCP_MODE);
if (rv == SMCOM_OK) {
rv = smGetSw(pApdu, &isOk);
if (isOk) {
rv = smApduGetResponseBody(pApdu, responseData, responseDataLen);
}
}
FreeAPDUBuffer(pApdu); return rv;
}

So this code is building an apdu_t data type (application protocol data unit), which is defined as follows

typedef struct
{
U8 cla;
U8 ins;
U8 p1;
U8 p2;
U8* pBuf;
U16 buflen;
U16 rxlen;
U8 extendedLength;
U8 hasData;
U16 lc;
U8 lcLength;
U8 hasLe;
U16 le;
U8 leLength;
U16 offset;
} apdu_t;

The interesting part here is pBuf and how it is manipulated, as this is where any payload data, such as the applet name, will go. Here is the implementation of this function:

// defined elsewhere:
define MAX_APDU_BUF_LENGTH 1454
// defined in sm_apdu.c:
ifndef USE_MALLOC_FOR_APDU_BUFFER
static U8 sharedApduBuffer[MAX_APDU_BUF_LENGTH];
endif
U8 AllocateAPDUBuffer(apdu_t * pApdu)
{
// AV: again, no null pointer checks.
assert(pApdu);
// In case of e.g. TGT_A7, pApdu is pointing to a structure defined on the stack
// so pApdu->pBuf contains random data
#ifdef USE_MALLOC_FOR_APDU_BUFFER
pApdu->pBuf = (U8*) malloc(MAX_APDU_BUF_LENGTH);
#else
pApdu->pBuf = sharedApduBuffer;
#endif
return 0;
}

Here we see that there are two options depending on whether dynamic allocation is supported on the target platform. If it is, a buffer of 1454 bytes is allocated dynamically. If it isn’t, a static buffer is provided as part of the firmware image for use.

Now if we go back to GP_Select, we see that it calls smApduAppendCmdData to add the applet name to the apdu. Let’s look at the details of this:

U16 smApduAppendCmdData(apdu_t *pApdu, const U8 *data, U16 dataLen)
{
// If this is the first commmand data section added to the buffer, we needs to ensure
// the correct offset is used writing the data. This depends on
// whether the APDU is a standard or an extended APDU.
if (pApdu->hasData == 0)
{
pApdu->hasData = 1;
ReserveLc(pApdu);
}
pApdu->lc += dataLen; // Value
memcpy(&pApdu->pBuf[pApdu->offset], data, dataLen);
pApdu->offset += dataLen; // adapt length
pApdu->buflen = pApdu->offset;
return pApdu->offset;
}

Here again, we have a problem. We know that pBuf is 1454 bytes, but dataLen can be set up to 65536 bytes. memcpy will copy regardless of the value of any byte, unlike strcpy, which terminates on null characters. The only difference here is that we will overwrite either the heap, or somewhere higher than the allocated location of the static buffer, depending on the configuration at compile time.

The most obvious question is: how exploitable is this? The answer really is: it depends. In the use case we were considering, the applet name was always the hardcoded string “a71ch”, although NXP code often derived the length of this using strlen. It would therefore require an attacker to overwrite these values in order to achieve any meaningful exploit.

In more complicated scenarios, this could easily be exploitable. This is especially true as in an embedded context, the many exploit mitigations present in modern operating systems do not exist; the operating system itself may be very barebones and all code is running in ARM’s supervisor context (equivalent to x86’s “ring 0”).

We also noted a number of “assert” null pointer checks during this review. These are “no-ops” unless code is built specifically to be debugged and so do not perform any kind of safety check at all. Again, if you are familiar with computer security you will already understand why this is a problem; if not, the quick explanation is that null pointers indicate memory at “address 0”. An attacker can control a program simply by writing to this well-known address, using a condition where a null pointer is present. Most modern desktop operating systems deliberately allocate the null page, with permissions set to deny any kind of access. This causes applications to crash when access using a null pointer is attempted. Embedded systems may not include such defences.

On behalf of our customer, we reported all of these issues to the vendor NXP and the issues have since been fixed in the latest 1.06 build of the host library. Firstly, in smApduAppendCmdData a maximum payload length is computed

// The maximum amount of data payload depends on (whichever is smaller) 
// - STD-APDU (MAX=255 byte) / EXTENDED-APDU (MAX=65536 byte)
// - size of pApdu->pBuf (MAX_APDU_BUF_LENGTH)
// Standard Length APDU's:
// There is a pre-processor macro in place that ensures 'pApdu->pBuf' is of sufficient size
// Extended Length APDU's (not used by A71CH):
// APDU payload restricted by buffersize of 'pApdu->pBuf'
U16 maxPayload_noLe;
if (pApdu->extendedLength) {
maxPayload_noLe = MAX_APDU_BUF_LENGTH - EXT_CASE4_APDU_OVERHEAD;
}
else
{
maxPayload_noLe = APDU_HEADER_LENGTH + APDU_STD_MAX_DATA;
}

Armed with this information, the length is later checked:

// Value
if (dataLen <= (maxPayload_noLe - pApdu->offset))
{
memcpy(&pApdu->pBuf[pApdu->offset], data, dataLen);
pApdu->offset += dataLen;
}
else
{
return ERR_INTERNAL_BUF_TOO_SMALL;
}

Which prevents data being copied into the buffer unless it is within bounds. The deprecated code has been removed entirely.

We believe this should serve as a cautionary tale about the state of security of embedded devices, or “IoT”. With a lot of old code in use, and code written in memory-unsafe languages such as C, it is difficult to build high-assurance hardware. Indeed, it is very easy for even the best of programmers to write code containing such bugs.

This post does however have a silver lining. We have only good things to say about NXP’s response to our submission. NXP patched and issued a new release well within the standard 90 day disclosure deadline, and kept us updated throughout the process. NXP PSIRT’s response should be held up as an example of how companies should accept security reports

Details:

  • Affected version: 1.05 (prior versions may be affected).
  • Fixed version: 1.06
  • Reported to NXP: 2019-02-25
  • Fixed (software available): 2019-03-18
  • Software download: here.

Releasing mqttinfo and scanning all open brokers with BinaryEdge

Today we’re releasing mqttinfo, a command-line utility we wrote to collect information on the configuration and behavior of a remote MQTT broker service. mqttinfo also attempts to guess the product type of the underlying software, in the same spirit as nmap’s OS detection – but with fewer and simpler heuristics at the moment.

mqttinfo is available at https://github.com/teserakt-io/mqttinfo in version 0.1.0, with almost no documentation, probably dozens of bugs, but we hope it’ll be useful nonetheless to MQTT users! We’ll do our best to fix any bugs found and to add new features.

As an example, here’s the output of mqttinfo when analyzing the open test server of the popular open-source broker mosquitto:

As you can see, mqttinfo supports MQTT versions 3.1.1 and 5, which are respectively the most common version and the recently standardized one. In this example, mqttinfo directly connects without any authentication because the service is open, but you can also pass a username and password to analyze authenticated brokers, as documented:

In the above example, the option -j will tell mqttinfo to append the results in JSON to a file ./mqttinfo.json, for example:

{"Host":"test.mosquitto.org","Port":1883,"Username":"","Password":"","V4":true,"V5":true,"V4Anonymous":true,"V4PublishSYS":true,"V4FilterSYS":true,"V4SubscribeAll":true,"V4InvalidTopics":false,"V4InvalidUTF8Topic":false,"V4QoS1":true,"V4QoS2":true,"V4QoS3Response":false,"V5Anonymous":true,"V5PublishSYS":false,"V5FilterSYS":true,"V5SubscribeAll":true,"V5InvalidTopics":false,"V5InvalidUTF8Topic":false,"V5QoS1":true,"V5QoS2":true,"V5QoS3Response":false,"TypeGuessed":"mosquitto","Failed":false,"Error":""}

It’s of course interesting to analyze one broker service, for example to detect misconfigurations or shortcomings of the broker you’ll be using in production. But it’s even better to analyze many brokers, such as all brokers available on internet. To do this efficiently and reliably, we worked with our friends at BinaryEdge, who offered to help after they noticed mqttinfo’s teaser.

BinaryEdge deployed mqttinfo as a module of their scanning platform, and ran it on all the hosts with port 1883 open (the port typically used for MQTT unencrypted connections). The scan took several hours—notably because mqttinfo’s software detection can take up to 20 seconds—and the main observations are the following:

  • Of all the hosts with TCP port 1883 open, 23287 authorized an anonymous connection over MQTT v3.1.1, by sending back a CONNACK packet with reason code 0x00. Many of these brokers may be there for testing and/or open on purpose, but many are evidently used by real devices, as BinaryEdge had already observed.
  • Of these 23287 hosts, about 70% seemed to be running Mosquitto as a broker—but keep in mind that our product detection is based on heuristics and can be easily fooled, so take this number with a grain of salt.
  • 71,5% of the hosts authorized clients to publish to $SYS topics (a behavior that we discussed in a previous post), a number only slightly greater than that of the mosquitto, and indeed almost all hosts identified as mosquitto share this property.
  • 77.5% of the hosts accepted an invalid UTF-8 string as a topic, whereas the standard states that “If a Server or Client receives a Control Packet containing ill-formed UTF-8 it MUST close the Network Connection”. We used the string \xc3\x28 to test this, and of course could not test all invalid UTF-8 strings.
  • 2.3% of the hosts accepted the invalid topic “A+”, which includes the wildcard character “+” in a way that is invalid, as per the standard.
  • A bit less than 1% of the hosts did not seem to support QoS level 1, with a similar number of instances not supporting QoS2. Surprisingly, about 4.5% of the broker instances (1055) accepted a QoS level of 3, which is not defined by the standard and should be rejected as invalid.
  • 88.5% of the hosts authorized clients to subscribe to all topics at once, by subscribing to the topic filter “#”.
  • Only about 10% of the hosts (2430) supported MQTT v5, of which only approximately 1570 seemed to support QoS levels greater than 0.

We’ve been running mqttinfo on several versions of the main broker products, and have often noticed different behaviors, not only across products, as expected, but also across versions. Furthermore, and in part due to differences in the norms defined by the standards, MQTT v3.1.1 and v5 interfaces may also behave differently even in a same version of a broker.

We have reported some of our observations to broker product maintainers, who often helped us understand MQTT better and sometimes corrected our understanding. We’d like to thank in particular HiveMQ and VerneMQ for their insightful feedback.

Last but not least, we’d like to thank Tiago from BinaryEdge and his team for offering to extend their worldwide scan of MQTT brokers with mqttinfo, and for rapidly providing us with the results!

Is MQTT secure? (A report)

Today we publish our report “Is MQTT secure?”, available here as a PDF, or directly below. (Such a report ought to be called a white paper, but having lost much of its prestige – and meaning – lately, that term didn’t sound right to us.)

When someone asks if something is secure it is not always easy to answer. People want a yes or a no even if they are incapable of explaining what “secure” means in the context. The last thing they want to hear is “it depends” – yet that’s often the least wrong answer.

This post elaborates on what “it depends” involves in the context of MQTT’s security, with a focus on confidentiality. Although we will focus on MQTT here, the same ideas apply to any protocol or communication between devices. Let’s start with some general background on the notion of security for online services.

“Secure”?

Imagine if a group of people are communicating using a social network or chat program, how then might we define security? There are a number of ways to look at this problem:

  1. Security of the service itself. Does the social network or chat server have any logic errors or exploitable code that might allow an unauthenticated user, or authenticated user not part of the group, to interfere with the messages of the group?
  2. Transport security to the service. Most internet packets take multiple hops between a user and the destination server, whether inside the destination social network’s network or outside of it. Can we trust these parties not to interfere with packets? Might they record them? Might other parties insert themselves into this path and do the same?
  3. End-to-end security. This ensures confidentiality and integrity between the communicating participants such that no intermediaries, even the social network or chat server, can view or tamper with communications. All they know are who is communicating with whom.
  4. Sender and recipient untraceability. This is where we do not know who is sending to whom at all, a.k.a. anonymity. We list this for completeness, but it’s hard to implement effectively and for this reason rarely guaranteed. The obvious real world example of this is Tor.

So how do common service providers fare when considering these security criteria?

Most services provide transport security through the use of TLS (you might know this as SSL – TLS is the protocol that has replaced SSL but the names are often used interchangeably). Whether you upload a photo on Facebook or Google services, or chat using Slack, you are doing this via TLS and subject to TLS being secure, so are your communications. Since the mass surveillance revelations by Edward Snowden from 2013 onwards, there has been an increasing push to deploy encryption and in particular TLS: Chrome now marks unencrypted sites as insecure, there’s Let’s Encrypt, “end-to-end encryption” has a Wikipedia page; Snowden made crypto jump from subculture to mainstream, which is probably good, all things considered.

The security of online service providers is, however, never a total guarantee (although we know they employ word class engineers and teams to guarantee the best level of protection they can). Things are even worse when you move away from companies that understand the internet. Most penetration testers can tell you horror stories of out of date software, poorly written and possibly outright insecure code, lack of hardening and no process to manage this for critical production software.

TLS’ limitations

Transport security is a great goal, but it cannot protect you if the security of the service in question is compromised, or if some of the service’s operators decide to look into your data. So point 2 (transport security) does not mitigate point 1 (service security). Specifically, suppose we have the following sequence of events:

  1. User Alice uploads a photo to a social networking website. Her browser negotiates TLS with the service and her photo is secured in transit.
  2. The service stores the photo unencrypted in one of their data centres.
  3. User Bob wishes to view the photo Alice just posted – perhaps it was a skiing trip to Verbier? He requests the photo from the service and being authorized by Alice already, he receives the photo. His browser likewise successfully negotiates and uses TLS.

We can visualize this in the following diagram:

The communication from Alice to the service and the service to Bob is secure, but the service itself has an unencrypted copy of Alice’s photo. So if a logic error exists, or the service is otherwise compromised, that photo can be extracted by the attacker.

End-to-end security comes in to play at this stage: leaving out the details, if Alice encrypts a photo for Bob, then the service in question can no longer view the photo. They can only disrupt its transmission, as the diagram below shows:

The machine-to-machine case

In the most popular machine-to-machine (M2M) protocol, MQTT, messages are sent to a server, called the broker, before being forwarded to their recipient(s). Most broker software support TLS, so you can get transport security today. This is excellent news and ensures that messages cannot be read by intermediate internet nodes.

What TLS does not protect against is, of course, the compromise of the broker. This could happen for a whole number of reasons, from outright logic errors in the software to misconfiguration by the user. For example, a vulnerability such as the one we found in a popular broker software may be leveraged to get access to the host running the broker, thereby granting the attacker’s access to all MQTT messages processed by the broker.

As stated above, TLS also leaves data exposed to operators of the broker, whom may not be trustworthy enough for critical applications – for example, if access to a confidential data stream is worth $1M to a competitor, said competitor could bribe or blackmail an operator for a fraction of this cost. Corporate espionage is a very real risk for many companies today and has never been easier. Companies that manage critical national infrastructure are particularly at risk.

Even worse, some devices rely on publicly managed brokers. When sharing a broker with other service providers in this way, while all information may be secure when talking to the broker, the only thing preventing other clients accessing your devices’ information is the access control implementation on the broker. If such a public broker is compromised or implements access control poorly, those messages can easily be read and modified. As we’ve recently observed, access control are not always safe by default, which increases the risk.

Increasingly, vehicles are becoming connected devices too. The data shared by vehicles might range from engine performance information that is of interest to competing manufacturers, to personal information such as owner location and the journeys they make. This information almost certainly comes under the purview of ever stringent data protection law (for example the GDPR).

This is why there is a need for end-to-end security in the M2M, V2X, and IoT spaces. As increasingly amounts of sensitive data are sent between autonomous nodes, such as vehicle locations or presence in private homes, it becomes even more important to ensure that even in the event of relay, proxy or broker compromise, only the intended recipients of those messages can read the data.

Intermission – MQTT & TLS

As far as confidentiality is concerned, MQTT only briefly mentions using TLS. The standard says: “The MQTT protocol requires an underlying transport that provides an ordered, lossless, stream of bytes from the Client to Server and Server to Client.”, and then states that, although the underlying transport is TCP/IP, TLS and WebSocket are “also suitable”.

In the MQTT standard’s Security section, we read that “it is strongly recommended that Server implementations that offer TLS SHOULD use TCP port 8883”. However, the standard does not recommend – let alone strongly – that TLS be used. Instead, the same section notes that “[as] a transport protocol, MQTT is concerned only with message transmission and it is the implementer’s responsibility to provide appropriate security features. This is commonly achieved by using TLS.” Fair enough.

(The above excerpts are taken from MQTT 3.1.1’s specification, but the latest one, 5.0, includes almost identical text.)

Cryptography of things

Implementing state-of-the-art cryptography in resource-constrained environments can present a technical challenge. In some constrained environments, the device may not be powerful enough to run public-key cryptography and/or to include a TLS implementation. Such environments can for example include some RFID chips or 8-bit AVR processors. But most of the time you’ll have symmetric cryptography algorithms such as AES and SHA-256. Yet crafting a protection scheme compatible with the system constraints isn’t always straightforward – for example, how do you add authentication when you can’t extend the payload size?

In more extreme cases – legacy hardware, high throughput constraints, and so on – standard crypto may still be insufficient. For such cases, NIST is running a competition that aims to standardize so-called lightweight cryptography algorithms. Note that the niche field of lightweight cryptography is all but new, and that many lightweight crypto schemes already exist; they’re just not blessed by NIST. (Fun fact: the MSc and PhD theses of a Teserakt founder were about lightweight crypto, back in 2006 and 2009, before it was cool!)

Protecting messages with cryptography is generally a solved problem, and in our experience the trickiest part is the case-specific engineering problem. Specifically, a non-trivial problem is to create a system that remains safe even if the device is partially compromised, if its pseudorandom generator fails, or if its clock is not accurate, yet consumes minimal energy and does not incur significant performance hits.

The hard part

In the same way that you can’t solve the halting problem, or that you can’t generate randomness deterministically, you can’t create trust with cryptography, no matter how much cryptography you use – to cite Jon Callas. In machine-to-machine networks, trust often simply consists in the association between a device identity and a cryptographic key. From there, we’ve got two main classes of problems related to key management:

First, problems that relates to key provisioning and trust: How do unique keys end up on devices? At what stage of the production chains are they generated and provisioned to the device? Can a device generate its own keys? How would they share their public or secret keys with other parties? What trust model should be used, pre-shared keys, trust-on-first-use, centralized and hierarchical PKI, or P3KI‘s decentralized model? There are no generic good answers to these questions – again, it depends.

Second, how do you encrypt, or more generally, how do you protect messages once every devices has its key? How are session keys derived from identity keys? Should there be a notion of session at all? How can you ensure that previous communications won’t be compromised if a key is compromised later (forward secrecy)? What about the opposite (backward secrecy – if a current key is compromised can we recover and protect future messages)? How do you realize secure group messaging among large sets of devices, as opposed to humans? How can you replace humans’ operations, and how can you leverage the absence of humans to better automate things? These questions and many others are mostly unexplored territory, and are our focus at Teserakt. We don’t claim to have all the right answers (we definitely don’t), but are doing our best to find the right trade-offs to provide the highest security at the lowest cost.

Is MQTT secure?

After reading the above you’ll now realize that the question is not the right one to ask. MQTT is not insecure, but it’s not secure, because it’s not designed to be secure. A slightly better question to ask could be “Are MQTT broker services secure?”, but then again it depends. A better question would be “Are MQTT broker services, and their related software, secure enough for application X?”.

An even better question would be “Do you have enough trust in the broker software, in the organization that runs it and its personnel, in said organization’s ability to perform security updates (not only of the broker but also of related software, such as hypervisors) to leave all your messages exposed to espionage and tampering today and a few years from now?”

Many organizations will acquiesce, and rightfully so, noting that they care more about availability than confidentiality and integrity. That could for example be the case of weather measurement data.

But many other organizations will reply that no, they think that risk is too high to be neglected or accepted. This might be the right answer when data transmitted relate to private individuals’ activity, to critical infrastructure systems, to safety monitoring systems, to proprietary technology (which may be reverse engineered only from metadata), to software updates, to geolocation information, and many other types of information that needs be protected to ensure a sustainable business.

The future

We would like to think the internet was always ready for e-commerce and that the cryptography and security of systems protecting it have always been fit for purpose. However, the truth is that our understanding of secure systems and our expectations have evolved over time. SSL Version 1 never made it outside of Netscape and SSH version 1 is considered badly broken. We have iterated these protocols over the last several decades to arrive at today’s level of transport security.

Security in personal messaging has likewise evolved from obscure, niche origins (such as OTR over what was originally called Jabber). The first attempt to bring end-to-end security to this space was TextSecure and RedPhone around 2012, a time in which all other messaging applications relied only on transport security. These applications were merged into Signal. Following the success of Signal, the protocol was licensed by and integrated into WhatsApp – among others – and is now used daily by around 1.5 billion people. Alternative products such as Wire, Telegram, and Facebook Messenger either launched with similar security standards, or rolled them out on their platforms.

Teserakt’s vision is that the same evolution is happening in the embedded, IoT, V2X, and M2M spaces. Today, messaging protocols such as MQTT and Kafka are at best providing transport security. We want the state of the art of end-to-end-security to apply to the smart home, to critical national infrastructure, to the cars you drive, and to the medical devices you use.

Solving the hard problem with E4

If you need end-to-end security, you should contact us to try our product, E4, which we believe is the best option today for integrating solid encryption and key management in IoT systems. We have solved the hard problems so that you don’t have to.