In this guide
Your ESP32 reads a temperature sensor. Now what? You need that reading to reach a dashboard, a phone app, or a database — reliably, over flaky WiFi, without draining the battery. HTTP can do it, but it's chatty: headers, handshakes, and polling add up fast on a constrained device.
MQTT is the protocol the IoT world standardized on for exactly this problem. It's lightweight, it's built around a publish/subscribe pattern that decouples your devices from your applications, and it handles unreliable networks gracefully. This guide explains how MQTT works, the concepts you must understand (topics, QoS, retained messages, last will), and how to use it in a real student project.
What MQTT is and why IoT uses it
MQTT (Message Queuing Telemetry Transport) is a messaging protocol designed for constrained devices and low-bandwidth networks. It was created in 1999 for monitoring oil pipelines over satellite links — the constraints were extreme, and the design reflects it: a minimal fixed header of just 2 bytes, a persistent TCP connection, and a broker in the middle.
The key architectural idea is publish/subscribe. Devices don't talk to each other directly, and they don't call your server's API. Instead:
- A device publishes a message to a topic (e.g.
home/livingroom/temperature). - Any client that subscribed to that topic (or a pattern matching it) receives the message.
- A broker sits in the middle, routing messages from publishers to subscribers.
Publishers and subscribers never know about each other. Your ESP32 publishes sensor data without caring whether zero or fifty dashboards are listening. Your web dashboard subscribes without knowing which device sent the data. This decoupling is what makes MQTT scale from one sensor to ten thousand.
Topics: the addressing system
Topics are UTF-8 strings with levels separated by forward slashes. They form a hierarchy you design:
home/livingroom/temperature
home/livingroom/humidity
home/bedroom/temperature
factory/line1/machine3/vibration
Subscribers use wildcards to match multiple topics:
+matches exactly one level:home/+/temperaturematcheshome/livingroom/temperatureandhome/bedroom/temperature, but nothome/livingroom/sensor1/temperature.#matches any number of levels (must be the last level):home/#matches everything underhome/.
Design your topic hierarchy early. A common student mistake is flat topics like temp1, temp2 — they become unmanageable the moment you add a second device. Hierarchical topics with wildcards let one dashboard subscription (factory/#) monitor an entire deployment.
Convention: avoid leading slashes (
/home/temp) — they create a confusing empty first level. Use lowercase, and keep topics descriptive but short since every byte counts on constrained links.
QoS: choosing your delivery guarantee
MQTT offers three Quality of Service levels. This is a per-message choice, and picking correctly matters:
| QoS | Name | Guarantee | Overhead | Use for |
|---|---|---|---|---|
| 0 | At most once | Fire and forget — may be lost | Minimal | High-frequency sensor data where one missed reading doesn't matter |
| 1 | At least once | Delivered, but may arrive duplicated | Moderate (acknowledgment) | Commands, alerts — where loss is unacceptable but duplicates are harmless |
| 2 | Exactly once | Delivered once, guaranteed | Highest (four-step handshake) | Billing, critical actuation — where duplicates would cause real harm |
QoS 0 is the right default for telemetry. If your temperature sensor publishes every 10 seconds and one reading is lost, the next one arrives 10 seconds later — no harm done. The overhead savings are real on battery devices.
QoS 1 fits commands: "turn on the relay." If the command is lost, nothing happens (bad). If it's duplicated, the relay just turns on twice (harmless). The subscriber must handle duplicates idempotently — for a relay command, "set state to ON" is naturally idempotent.
QoS 2 is rarely needed in student projects. It exists for cases like "dispense exactly one dose" where a duplicate would be dangerous. The four-step handshake costs battery and latency — don't use it by default.
Retained messages and the Last Will
Two features that solve real problems beginners hit:
Retained messages: normally, a subscriber only receives messages published after it subscribes. If your dashboard connects and subscribes to home/livingroom/temperature, it sees nothing until the next sensor reading arrives. Mark a message as retained and the broker stores the last one per topic — new subscribers immediately receive the most recent value. Use retained messages for state ("the relay is currently ON", "last known temperature"), not for event streams.
Last Will and Testament (LWT): when a device connects, it can register a "will" message with the broker: "if I disconnect unexpectedly, publish this." If the ESP32 loses power or WiFi drops, the broker publishes the will — e.g. home/sensor1/status: offline — to all subscribers. This is how dashboards show devices as offline instead of displaying stale data forever. Every battery or field device should set a LWT.
The broker: your options
The broker is a server that routes messages. For student projects:
- Mosquitto — the standard open-source broker. Runs on a Raspberry Pi, a VPS, or your laptop. Start here; it handles everything a student project needs.
- HiveMQ Cloud / EMQX Cloud — managed brokers with free tiers. Useful when you don't want to run a server, but check the free-tier message limits.
- AWS IoT Core — enterprise-grade, but the setup complexity and pricing model are overkill for most student work.
For a final-year project, Mosquitto on a cheap VPS or a Raspberry Pi on the campus network is the sweet spot: full control, no message limits, and you learn how the infrastructure actually works.
MQTT in practice: a minimal ESP32 example
Using the PubSubClient library (the standard Arduino MQTT client):
#include <WiFi.h>
#include <PubSubClient.h>
WiFiClient wifiClient;
PubSubClient mqtt(wifiClient);
void setup() {
// ... WiFi connection code ...
mqtt.setServer("broker.example.com", 1883);
mqtt.setCallback(onMessage); // function called for incoming messages
}
void onMessage(char* topic, byte* payload, unsigned int len) {
// Handle incoming commands, e.g. relay control
if (String(topic) == "home/livingroom/relay/set") {
bool on = (payload[0] == '1');
digitalWrite(RELAY_PIN, on ? HIGH : LOW);
// Publish the new state as retained so dashboards sync
mqtt.publish("home/livingroom/relay/state", on ? "1" : "0", true);
}
}
void loop() {
if (!mqtt.connected()) reconnect();
mqtt.loop(); // must be called frequently - handles keepalive and incoming
static unsigned long lastSend = 0;
if (millis() - lastSend > 10000) {
lastSend = millis();
float t = readTemperature();
char buf[8];
snprintf(buf, sizeof(buf), "%.1f", t);
mqtt.publish("home/livingroom/temperature", buf); // QoS 0, fine here
}
}
void reconnect() {
while (!mqtt.connected()) {
// Client ID must be unique per device!
// Last will: publish "offline" to status topic if we drop
if (mqtt.connect("esp32-livingroom-01", "user", "pass",
"home/livingroom/status", 0, true, "offline")) {
mqtt.publish("home/livingroom/status", "online", true); // retained
mqtt.subscribe("home/livingroom/relay/set"); // QoS 1 for commands
}
delay(5000);
}
}
Three things beginners get wrong in this pattern:
- Not calling
mqtt.loop()frequently. It handles keepalive pings and incoming messages. Block for 30 seconds indelay()and the broker will disconnect you for missing keepalives. - Reusing client IDs. The broker kicks the older connection when a duplicate client ID connects. Two devices with the same ID will fight forever. Use unique IDs (e.g. include the MAC address).
- No reconnection logic. WiFi drops. The
reconnect()loop above is not optional — it's the difference between a demo and a deployment.
MQTT vs HTTP: when to use which
| MQTT | HTTP | |
|---|---|---|
| Connection | Persistent (one TCP connection) | New connection per request (or keep-alive) |
| Pattern | Publish/subscribe (decoupled) | Request/response (coupled) |
| Overhead per message | ~2 byte header | Hundreds of bytes of headers |
| Server-to-device messages | Natural (subscribe) | Requires polling or WebSockets |
| Best for | Telemetry, commands, real-time | One-off requests, file upload, REST APIs |
Use MQTT for the device-to-cloud data path and device commands. Use HTTP for things like firmware downloads or one-off configuration fetches. Many real projects use both.
Security: don't skip this
A surprising number of student projects run MQTT with no authentication on a public broker. Don't:
- Always use authentication — username/password at minimum. Mosquitto supports it with a password file.
- Use TLS on untrusted networks — MQTT over TLS (port 8883) encrypts the connection. On an ESP32 this costs some memory and handshake time, but on any network you don't control it's non-negotiable.
- Never expose an unauthenticated broker to the internet. Bots scan for open MQTT brokers constantly. An open broker becomes a free message relay for strangers within hours.
Going further
Once you have basic pub/sub working, the next concepts are: MQTT v5 features (reason codes, message expiry, topic aliases), bridging brokers for multi-site setups, and integrating with time-series databases for storage. For the cloud side of your architecture, the guide to connecting IoT projects to the cloud covers the full picture. For battery-powered devices publishing over MQTT, read the ESP32 low-power design guide — the awake-window math directly determines your MQTT keepalive and publish strategy. More IoT concepts live in the IoT & Embedded branch hub.