Bee-Inspired Always-On Voice Companion: Edge Wake-Word Wearable that Summarizes Conversations (ESP32 + On-Device AI)

Amazon's October 2026 Bee launch put an always-on listening wearable on the map: a device that sits on your wrist or phone, hears your conversations and hands you back summaries, to-dos, reminders and insights, with a physical mic button and an LED as its privacy story. Students want to build that — but the commercial route runs on cloud audio and a monthly subscription, and no student can audit a cloud pipeline. The gap is a Bee that never phones home. This project is that build: a small disc-shaped BLE puck carrying an ESP32-S3, an INMP441 I2S digital MEMS microphone, a hardware mute slider and a status LED, paired with a companion phone app. A tinyML wake-word model on the puck gates capture; voice-activity detection discards silence; short speech snippets cross BLE to the phone, where offline transcription (Vosk), extractive summarization and date/time extraction turn them into a conversation timeline — summary, to-dos, reminders — stored in local SQLite. The honest math: BLE throughput carries brief snippets, not continuous stereo, so the phone stays nearby and on-device transcription trades accuracy against the cloud's. What you get is a working edge-AI voice companion whose entire data flow a student can trace, measure and defend.

1. Sound hits the INMP441's MEMS element — a microscopic silicon diaphragm whose deflection changes capacitance, digitized on-chip — and leaves as a 24-bit I2S stream into the ESP32-S3's hardware I2S peripheral.
2. Voice-activity detection on one LX7 core rejects silence; the tinyML wake-word model on the other core continuously scores the stream for the "hey hive" phrase, with the threshold configurable in firmware.
3. On a wake-word match, the puck opens a BLE link and sends short speech snippets (a few seconds each) to the paired phone; BLE throughput is the hard limit, so this is brief clips, never continuous stereo.
4. The companion app receives each snippet, transcribes it offline with Vosk, then runs extractive summarization to compress the conversation to its key points.
5. Date/time pattern matching scans the transcript for reminders ("tomorrow 10 AM") and action phrases become to-dos; the card is written to local SQLite with a timestamp and a location tag.
6. The timeline UI renders the summary, chips and transcript snippet, and the privacy tab mirrors the puck's mute-slider and LED state; with the slider closed, the mic rail is unpowered and capture is physically impossible.

Tech stack:

• ESP32-S3 (dual-core Xtensa LX7 · BLE + on-puck inference)
• INMP441 I2S digital MEMS microphone
• TinyML wake-word model (Edge Impulse-style workflow)
• Voice-activity detection (on-puck gating)
• Vosk offline speech recognition (small English model)
• Extractive summarization (TextRank-style) + regex date/time extraction
• Companion phone app + local SQLite storage
• 3.7 V LiPo + TP4056 charge module (puck power)

• [TinyML wake-word detection] An on-device wake-word model ("hey hive", configurable) trained through an Edge Impulse-style workflow runs continuously on one ESP32-S3 core, so the puck only opens its BLE audio path after the phrase — nothing streams until the wake word lands.
• [Voice-activity gating] A voice-activity detector on the second core discards silence and non-speech frames before they ever reach the radio, keeping BLE traffic to real speech windows and stretching battery life.
• [INMP441 I2S digital microphone] The mic delivers 24-bit digital audio straight over I2S (61 dBA SNR, 60 Hz–15 kHz, −26 dBFS sensitivity — datasheet): no analog stage, no codec, no preamp noise path to debug.
• [Hardware mute slider] A physical slide switch sits in series with the microphone supply. Closing it cuts mic power at the circuit level — firmware cannot override it, and the app reports the muted state.
• [Mic-rail status LED] An amber LED wired to the microphone power rail, not to a GPIO: lit means the mic is powered, dark means it physically cannot be listening. Privacy you can verify with a multimeter.
• [Offline transcription] Received snippets are transcribed on the phone with the Vosk small English model (~40 MB), entirely offline. Accuracy is below cloud services — stated honestly — but no audio is ever uploaded.
• [Summaries, to-dos and reminders] An extractive TextRank-style summarizer condenses each conversation; date/time pattern matching pulls out reminders and action phrases as to-dos. Everything is computed on the phone.
• [Local SQLite timeline] Every conversation card — summary, transcript snippet, to-dos, reminders — is stored in an on-phone SQLite database. There are no accounts, no servers and no sync, so the privacy architecture is the absence of a data path, not a promise.

• Assembled listening puck: ESP32-S3, INMP441 mic, mute slider, status LED, LiPo + TP4056, 3D-printed disc case
• Complete puck firmware source (Arduino IDE / ESP-IDF style) with wake-word threshold tuning guide
• Companion phone app source with the conversation timeline and privacy panel (demo UI included)
• Wake-word training workflow notes (Edge Impulse-style pipeline, configurable phrase)
• Wiring diagram and puck assembly layout
• 3D-print files for the disc case with mic-grille pattern
• Component list with Indian supplier sourcing notes for the mic, S3 board and LiPo
• Project report PDF (I2S audio path, tinyML pipeline, BLE limits, privacy architecture)
• PPT presentation for final review
• Viva Q&A preparation document (wake-word models, I2S vs analog mics, on-device vs cloud trade-offs, BLE throughput)