Offline Voice Assistant Project using Python and Vosk Speech Recognition

Cloud voice assistants send every utterance to a server: they need constant internet, they leak private audio, and they cannot run on a locked-down lab machine. Students who want to understand speech interfaces are left with two bad options — a toy that fakes it with typed input, or a cloud API key they do not control. The offline alternative is real: Vosk, the open-source speech-recognition toolkit from Alphacep, decodes audio entirely on-device with a compact downloadable model (the English small model is only about 40 MB). This project builds a complete voice assistant around it — PyAudio captures microphone audio at 16 kHz, Vosk turns it into text, a deterministic regex intent engine extracts the action and its slots, handlers execute it, and pyttsx3 speaks the answer back. Because every stage is local and inspectable, the student can explain the full pipeline in a viva, from PCM frames to spoken reply.

1. PyAudio opens the microphone at 16,000 Hz, 16-bit mono — the exact PCM format Vosk expects — and streams raw audio frames.
2. vosk.KaldiRecognizer, loaded with the downloaded vosk-model-small-en-us-0.15, converts the frames to text on-device; partial results update live and a final transcript is emitted at end of utterance.
3. The transcript is lowercased and normalized, then checked against custom commands.json triggers first, then the 12 built-in regex rules.
4. The matched intent's handler runs: reading the clock, evaluating an arithmetic expression, scheduling a reminder or timer, or returning a labelled stub reply for weather/music.
5. pyttsx3 converts the reply text to speech through the system TTS engine and plays it on the speakers.
6. The transcript, intent name, extracted slots and reply are appended to the timestamped session log.
7. The intent test set (pytest) replays a list of spoken-style commands through the parser to measure intent-classification accuracy for the report.

Tech stack:

• Python 3, Vosk (Alphacep) + vosk-model-small-en-us-0.15 (approximately 40 MB)
• pyttsx3 offline text-to-speech
• PyAudio microphone capture (16 kHz, 16-bit mono)
• Pure-Python regex intent engine (no cloud, no ML framework needed)
• commands.json custom-command configuration
• pytest intent test set
• HTML/CSS/JS command-console demo (mirrors the Python engine)
• Git

Dataset & model details

• Dataset / model: vosk-model-small-en-us-0.15 — open-source English (US) speech-recognition model published by the Vosk authors (Alphacep), approximately 40 MB download from alphacephei.com/vosk/models. The acoustic model is a third-party component; the student's own engineering is the intent engine, handlers, TTS wiring and evaluation.
• Task: (1) speech-to-text — 16 kHz mono audio in, English transcript out; (2) intent classification — transcript in, one of 12 intents plus extracted slots out; (3) spoken response — reply text in, system-TTS audio out.
• Model: Vosk KaldiRecognizer (16,000 Hz sample rate) for decoding; deterministic regex rule engine for intent parsing (custom triggers first, then built-in rules); pyttsx3 for speech synthesis.
• Metrics: intent-classification accuracy measured by the student on a self-built spoken-command test set using the included pytest runner (buyer-run procedure, template test list included). No word-error-rate figure is claimed for the acoustic model — it is reported as-is from the Vosk release, and the report states the achieved intent accuracy from the student's own test run.

• [Offline speech recognition] Vosk (Alphacep) decodes microphone audio fully on-device using vosk-model-small-en-us-0.15 (approximately 40 MB download) — no account, no API key, no internet at runtime.
• [Deterministic intent engine] A pure-Python regex rule engine maps each transcript to one of 12 built-in intents plus slots (for example, "remind me to call mom in 10 minutes" becomes set_reminder with task and time slots) — explainable line by line in a viva.
• [Offline text-to-speech replies] pyttsx3 speaks every reply through the operating system's own TTS engine, so the assistant talks back with zero cloud calls.
• [Custom command builder] Trigger phrases and replies live in a commands.json file and are matched before built-in rules — the student demonstrates extensibility by adding their own commands.
• [Reminders and timers with spoken alerts] The assistant confirms each reminder aloud and announces it when due; the console demo shows the same behaviour live.
• [Timestamped command log] Every transcript, parsed intent and reply is logged with timestamps, giving the report real run evidence instead of screenshots alone.
• [Optional: app launching] The reference build can shell out to launch desktop apps (calculator, notepad, browser) — kept optional so the core stays fully offline.
• [Future scope: wake word] Continuous listening with a wake word (for example via an open-source wake-word engine) is documented as future scope, not claimed as shipped.

• Complete Python source: listener, Vosk decoding loop, intent engine, action handlers, pyttsx3 reply wiring
• requirements.txt and the one-command model download/setup script
• commands.json custom-command system with trigger-first priority
• Intent test set with pytest runner and a template list of spoken-style test commands
• Working command-console demo (single-file HTML) mirroring the Python intent engine, with pipeline and custom-command views
• Project report PDF (background, Vosk pipeline, intent-engine design, methodology, evaluation)
• PPT presentation for final review
• Viva Q&A preparation document (speech recognition basics, Vosk decoding, regex intent parsing, offline TTS, evaluation)