Encrypted Mosquitto MQTT broker in Kubernetes

Eclipse Mosquitto is an open-source MQTT broker supporting MQTT(S) and Websocket (WS), popular for applications such as IoT.

An official Docker image is available on Docker Hub, making it possible to deploy to Kubernetes.

Public-facing services require encryption which implies SSL certificates. For HTTP-based applications deployed in Kubernetes, SSL certificates are commonly handled by cert-manager with a certificate issuer such as Let’s Encrypt. This makes it possible to encrypt WS connections but encrypting MQTT is more involved.

This article presents how to deploy an encrypted Mosquitto broker, supporting both MQTTS and WSS connections, to Kubernetes.

Prerequisites

For the purpose of this article, a public-facing Kubernetes cluster with an ingress controlle and rcert-manager installed is required. For the following, the domain example.com and subdomain mqtt.example.com are used as an example and needs to be replaced accordingly.

Deployment

Running a Mosquitto container in Kubernetes can be achieved by creating a Kubernetes deployment. The following manifest creates such deployment with ports 8883 and 9001 exposed:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mosquitto
spec:
  replicas: 1
  selector:
    matchLabels:
      app: mosquitto
  template:
    metadata:
      labels:
        app: mosquitto
    spec:
      containers:
      - name: mosquitto
        image: eclipse-mosquitto
        ports:
        - containerPort: 8883
        - containerPort: 9001
        volumeMounts:
        - mountPath: /etc/mosquitto/
          name: config
        - mountPath: /mosquitto/certs/
          name: certs
        - mountPath: /mosquitto/data/
          name: data
      volumes:
      - name: config
        configMap:
          name: mosquitto-config
      - name: certs
        secret:
          secretName: mosquitto-certs
      - name: data
        persistentVolumeClaim:
          claimName: mosquitto-data

Here, the pod specification configures three volumes mounts:

• /etc/mosquitto/: Directory holding mosquitto.conf, the Mosquitto configuration file, mapped to a ConfigMap
• /mosquitto/data: Directory holding the Mosquitto persistent data, mapped to a PVC
• /mosquitto/certs: Directory where SSL certificates are stored, mapped to a secret. This secret will be created by cert-manager (see below)

ConfigMap

When run as a container, Mosquitto is configured via the /etc/mosquitto/mosquitto.conf file. In Kubernetes, this file can be mounted as a ConfigMap such as the following.

apiVersion: v1
kind: ConfigMap
metadata:
  name: mosquitto-config
data:
  mosquitto.conf: |

    persistence true
    persistence_location /mosquitto/data/
    log_dest stdout
    
    # MQTTS listener
    listener 8883
    protocol mqtt

    cafile /etc/ssl/certs/ca-certificates.crt
    keyfile /mosquitto/certs/tls.key
    certfile /mosquitto/certs/tls.crt

    # WS Listener
    listener 9001
    protocol websockets

Here, two listeners are configured, matching the exposed ports in the configured in the deployment:

• An MQTT listener on port 8883 used for MQTTS connections
• A Websocket listener on port 9001 used for WSS connections

Moreover, Mosquitto is instructed to fetch SSL certificate and key files from the /mosquitto/certs/ directory. As specified in the deployment, this directory is mapped to a secret which is generated by cert-manager as explained further in the article.

On the other hand, the websocket listener does not require SSL configuration as termination will occur at the Ingress (see below).

An important point to notice is that, for the sake of simplicity, authentication settings have been left out. For a public-facing Mosquitto instance, it is imperative to add authentication to the broker.

PVC

As defined mosquitto.conf, data requiring persistence is set to be stored in /mosquitto/data/. According to the deployment, this directory is mapped to a PVC, which can created with the following manifest.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: mosquitto-data
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

Services

Mosquitto is exposed to clients via Kubernetes services. Here two services are created, one for each listener.

apiVersion: v1
kind: Service
metadata:
  name: mosquitto-mqtts
spec:
  type: NodePort
  selector:
    app: mosquitto
  ports:
  - port: 8883
    nodePort: 30883
---
apiVersion: v1
kind: Service
metadata:
  name: mosquitto-ws
spec:
  type: ClusterIP
  selector:
    app: mosquitto
  ports:
  - port: 9001

As MQTTS connections are encrypted directly at the Mosquitto container level, port 883 is exposed directly via a NodePort, mapped here to 30883. This will allow MQTTS connections to be established via mqtts://mqtt.example.com:30883.

On the other hand, WS connections are encrypted externally, which is handled with an Ingress as described in the following section.

Ingress

Providing external WSS connections to Mosquitto can be achieved with an ingress such as the following. With cert-manager and a certificate issuer such as Let’s Encrypt, SSL certificates can be automatically obtained at deployment, enabled WSS connections via wss://mqtt.example.com

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: mosquitto
  annotations:
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: letsencrypt
spec:
  tls:
  - hosts:
    - mqtt.example.com
    secretName: mosquitto-certs
  rules:
  - host: mqtt.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: mosquitto-ws
            port: 
              number: 8883

Here, SSL certificates are stored in Kubernetes as a secret named mosquitto-certs , which as described above, is mounted to /mosquitto/certs/ . This allows the certificates obtained by cert-manager for WSS connections to be also used for MQTTS.

Conclusion

By following this article, a encrypted Mosquitto instance can be deployed to Kubernetes, accepting encrypted connections on both mqtts://mqtt.example.com:30883 and wss://mqtt.example.com.