In this guide
Your ESP32 publishes sensor data. A Raspberry Pi on your desk runs the MQTT broker and the dashboard. It works — until the Pi's SD card dies the night before demo day, or your evaluator asks "but does it work from anywhere?" AWS IoT Core is Amazon's managed IoT cloud: the broker, device registry, and data routing run by AWS, reachable from anywhere, with a free tier that covers genuine student projects.
This guide explains what IoT Core actually gives you, the core concepts (things, shadows, the rules engine), how an ESP32 connects with certificates, what the free tier covers, and — honestly — when it is overkill and a simpler setup serves you better.
What AWS IoT Core is
IoT Core is a managed MQTT broker plus device management, built for fleets from one device to millions. Instead of running Mosquitto on a VPS, AWS runs the broker: you get a unique MQTT endpoint, TLS encryption, per-device certificates, and integrations that route device data into storage, alerts, and dashboards without you writing glue servers.
The honest positioning for students:
| Self-hosted Mosquitto | AWS IoT Core | |
|---|---|---|
| Setup | Install on Pi/VPS, configure yourself | Console clicks + certificates |
| Cost | Hardware/electricity you already have | Free tier, then per-message pricing |
| Reliability | Depends on your hardware | AWS-managed |
| Learning | Broker administration | Cloud IoT architecture |
| Scale story | "Runs on my Pi" | "Runs on AWS infrastructure" |
IoT Core shines when your project story involves the cloud: data accessible from anywhere, integration with databases and alerting, and a deployment narrative that sounds like industry practice. It is overkill when everything could stay local — a greenhouse monitor viewed on the farm does not need Virginia.
Core concepts
Things are entries in the device registry — the cloud's record that a physical device exists, with attributes ("this is the greenhouse node, firmware 1.2") and certificates attached. Creating a thing does not magically connect hardware; it creates the identity your hardware will use.
Certificates are how devices authenticate. Each device gets an X.509 client certificate plus a private key; the ESP32 presents the certificate during the TLS handshake instead of a username and password. Policies attached to certificates control what each device may do (publish to its own topics only, for example). Guard the private key — anyone holding it can impersonate the device.
Device Shadows are cloud-stored JSON documents holding each device's desired and reported state. The classic use: your app sets the shadow's desired state to {"pump": "on"}; the device, whenever it next connects, reads the shadow, turns the pump on, and reports back. Shadows decouple applications from device connectivity — commands wait patiently for offline devices instead of failing.
The Rules Engine routes messages with SQL-like statements: when a message arrives on a topic, a rule can filter it, transform it, and republish or send it to another AWS service (store in a database, trigger an alert, invoke a function). Rules are where "temperature exceeds 40°C → send notification" lives without a server you maintain.
Topics in IoT Core work like standard MQTT topics with reserved prefixes ($aws/things/...) for shadow operations.
Note: IoT Core's MQTT has AWS-specific behaviors — notably, persistent sessions and some QoS/retained-message semantics differ from Mosquitto. Test your exact patterns rather than assuming identical behavior.
Connecting an ESP32: the moving parts
The device side needs three things: the AWS IoT endpoint address (unique per account), the device certificate and private key files, and Amazon's root CA certificate. The firmware (using an MQTT library with TLS support, e.g. the AWS IoT Arduino SDK or PubSubClient with WiFiClientSecure) connects to port 8883 with mutual TLS — the server proves itself to the device with the root CA, the device proves itself with its certificate.
Practical advice from student builds:
- Certificates on the ESP32: store them in flash (PROGMEM or a filesystem partition), never hardcode them into public repositories. A leaked device certificate means revoking and reprovisioning.
- Time matters: TLS certificate validation needs correct time. The ESP32 must sync via NTP before connecting, or the handshake fails with confusing errors. This is the number-one "it works on Mosquitto but not AWS" cause.
- Memory: TLS on ESP32 costs significant RAM. Keep the firmware lean; if you hit allocation failures, the TLS buffers are the usual suspect.
- Policies, least privilege: the certificate policy should allow only what the device needs — publish to its telemetry topics, subscribe to its command topics, access its own shadow. The default "allow everything" policy is for testing only.
The free tier: what students actually get
AWS's free tier for IoT Core covers a meaningful allowance of messages and connection time monthly for the first year. For a final-year project — a handful of devices publishing every minute — you will stay comfortably inside it. The billing dimension to respect is messages: each publish counts, and chatty firmware (publishing every second "just because") is what pushes projects toward charges. Design sensible publish intervals and the free tier is plenty.
Set up a billing alarm on day one even if you expect to stay free. It costs nothing and it is the professional habit.
A sensible student architecture
The pattern that works for most final-year projects:
- ESP32 publishes telemetry to IoT Core topics (JSON, modest rate).
- A rules-engine rule filters and routes readings into cloud storage.
- Device shadows hold command state (desired vs reported).
- A lightweight dashboard or phone app reads storage and writes shadow desired-states.
Keep the device dumb and the cloud smart: the ESP32 senses and actuates; thresholds, history, and user management live in the cloud where they are easy to change without reflashing.
Rules engine patterns for student projects
The rules engine is where IoT Core stops being a broker and starts being a platform. Three patterns cover most student needs:
Filter and store. A rule watches the telemetry topic, keeps only messages matching a condition (readings from a specific device, or values outside a deadband), and writes them to cloud storage. This becomes your history database without a server — and the filtering keeps storage proportional to useful data rather than raw chatter.
Threshold alert. A rule watches for readings crossing a limit and republishes a compact alert message to a dedicated alert topic, which a notification service picks up. The device stays dumb (it just reports); the intelligence lives in the rule, changeable without reflashing.
Shadow synchronizer. A rule listens to shadow update events and mirrors the reported state into storage, giving you a queryable history of device state changes — when the pump turned on, what the setpoint was — alongside the raw telemetry.
Design guidance: keep rules simple and observable. Each rule should do one thing, log what it discards during development, and have a path for malformed messages. A silent rule that drops data is worse than no rule — you will not know what you are missing until the report needs that graph.
Common mistakes checklist
- Skipping NTP time sync. TLS fails without correct time — sync before connecting.
- Certificates in a public repo. Private keys in git history are compromised keys; use gitignore and provisioning scripts.
- Allow-everything policies in production. Scope each device to its own topics and shadow.
- Publishing every second. Message volume is the billing dimension — publish at the rate the application needs.
- No billing alarm. Set it on day one, even on the free tier.
- Assuming Mosquitto-identical behavior. Test retained messages, QoS, and sessions against IoT Core specifically.
- Using cloud when local would do. If nobody needs remote access, a local broker is simpler and free forever.
Where to go from here
- MQTT protocol explained with practical examples — the protocol underneath IoT Core: topics, QoS, retained messages, and last-will.
- ESP32 low-power battery design guide — the hardware companion for cloud-connected battery nodes.
- More cloud-connected builds in the IoT & Embedded branch hub.