EKA-1-Inspired Offline Multilingual Voice Assistant Project for Indian Languages

Most voice assistants in Indian homes only work while they can reach a cloud server, which fails in homes and hostels with patchy internet and in classrooms or labs where sending every spoken word to a server is not acceptable. The assistants that do work offline are usually English-only command matchers that recognise a fixed list of phrases and cannot hold a conversation. The IndiaAI Mission's launch on 3 November 2026 of EKA-1, a 26.3-billion-parameter foundation model covering all 22 scheduled Indian languages, and the open-source Dhir 2B and Dhir 5B models built for secure on-premise use, made Indian-language AI a national priority, but none can run on a student's desk. This project closes that gap honestly: a Raspberry Pi 5 voice-assistant device that listens, understands and answers in Hindi, Marathi and Tamil entirely offline, using a small quantized 2B-class language model with on-device speech recognition, plus a dashboard showing the conversation log, language picker and device status.

The mechanism runs from spoken word to spoken answer in five stages a student can trace and an examiner can test. The USB mic array captures audio and a small on-device recognizer listens only for the wake word "Namaste Pi", so the heavier stages stay asleep until the device is called. Once woken, Vosk transcribes the spoken sentence on the Pi, the text goes to the small quantized language model running through llama.cpp, and the model replies in the same language. Piper then turns that reply into speech and plays it through the speaker, while a light-on command travels as a local MQTT message to the relay. A wake word is only a short audio-pattern match, which is why the Pi can listen all day on one small recognizer while the 2B model stays asleep until the wake word fires, and why nothing is recorded or processed before the wake word.

1. The builder assembles the device: Raspberry Pi 5 in a case on a wooden base, USB mic array on top, speaker beside it, 3.5-inch status display at the front, and two relay channels wired to a light and a fan.
2. The builder installs the offline stack on the Pi: Vosk with one small speech model per demo language, Piper with matching voices, llama.cpp, and the 2B-class quantized language model in GGUF form, all downloaded once before the demo.
3. The builder starts the local MQTT broker on the Pi and connects the two relays, then checks from the dashboard that the light and fan switches respond.
4. For the conversation demonstration, the builder says the wake word, asks a question in Hindi, and shows the dashboard log recording the question and the spoken Hindi reply, then repeats the same exchange in Marathi and Tamil.
5. For the home-control demonstration, the builder says "light on" in one of the demo languages and shows the relay switching the light, then switches the fan the same way, with each action appearing in the dashboard log.
6. For the offline demonstration, the builder disconnects the Pi from the internet and repeats a full exchange, showing that the wake word, transcription, reply and spoken answer all still work with no network.
7. The builder exports the conversation log from the dashboard, points to the wake-word events, the language switches and the MQTT actions, and reports honestly in the report which exchanges succeeded and where recognition struggled.

Tech stack:

• Raspberry Pi 5 · 8 GB board for on-device inference
• USB microphone array · far-field voice capture
• Vosk offline speech recognition · on-device transcription
• llama.cpp · quantized model inference runtime
• 2B-class quantized language model · Dhir-2B-style open model
• Piper offline text-to-speech · Hindi and English voices
• MQTT local broker · home-device control messages
• Companion web dashboard · conversation log and device status

Dataset & model details

• Dataset: No training dataset is used: the build runs a pre-trained open-weight language model and pre-trained Vosk and Piper speech models, so there is no dataset to validate and no training run to report.
• Task: Spoken question in Hindi, Marathi, Tamil or English in, free-form text reply out in the same language, plus a device-control intent when the question is a command.
• Model: 2B-class open-weight language model in quantized GGUF form (Q4_K_M) run through llama.cpp, with Vosk for speech-to-text and Piper for text-to-speech, all on-device.
• Metrics: Task success on a scripted test phrase set in each demo language, response latency per reply, and wake-word misses per test session, all reported from the student's own demo runs, never claimed in advance.

How this build compares

This offline voice assistant is built around one promise: nothing spoken in the room ever leaves the room. A USB mic array listens for the wake word, Vosk transcribes speech on the device, a small quantized language model answers in the same Indian language, and Piper speaks the reply aloud, all without an internet connection. The companion dashboard makes the whole loop visible: the conversation log, the language picker, the wake-word status and the device controls. Two MQTT relay channels turn the assistant into something demonstrable rather than a talking box, switching a light and a fan on spoken command. The demo set is Hindi, Marathi and Tamil plus English, and every behaviour below is demonstrated in the same run, so the examiner never has to take a claim on trust.

• [Offline multilingual conversation] The device listens, understands and answers in Hindi, Marathi, Tamil and English with no internet at any point after setup: speech is transcribed on-device, the reply is generated on-device, and the answer is spoken on-device — implemented in the base build.
• [Wake-word listening] The mic array listens continuously for the wake word "Namaste Pi" using a small on-device recognizer, and the language model only starts after the wake word is heard, so the device is not recording or processing conversation before it is called — implemented in the base build.
• [2B-class quantized language model] A small open-weight language model of the 2B-parameter class, in the style of the open-source Dhir 2B model, runs in quantized GGUF form through llama.cpp on the Pi, giving free-form replies instead of fixed command matching — implemented in the base build.
• [On-device speech recognition and synthesis] Vosk handles speech-to-text and Piper handles text-to-speech, both fully offline, with a Hindi voice among Piper's supported voices, so the spoken answer arrives in the language the question was asked in — implemented in the base build.
• [MQTT home-device control] Two relay channels switch a living-room light and a ceiling fan from spoken commands, sent as local MQTT messages on the home network, giving the assistant a visible physical action to demonstrate — implemented in the base build.
• [Companion web dashboard] The dashboard shows the live conversation log, the language picker, wake-word status, model and device status, and the light and fan switches, so the whole offline loop is inspectable from a browser on the same network — implemented in the base build.
• [Small status display] A 3.5-inch display on the device shows the waveform, the active language and the listening state, so the device communicates what it is doing without needing the dashboard open — implemented in the base build.
• [Extra language packs] Future scope only, not implemented: adding further Indian languages by loading additional Vosk and Piper voice packs, listed as an extension rather than a claimed feature, since the base build ships the four demo languages.

• Complete fabricated voice-assistant device (Raspberry Pi 5 8 GB, USB mic array, speaker, 3.5-inch status display, two relay channels, wooden base, wiring)
• Offline software stack (Vosk speech models for the four demo languages, Piper voices, llama.cpp, 2B-class quantized language model, local MQTT broker, companion web dashboard)
• Setup and calibration guide (model download, language pack setup, wake-word test, relay wiring, offline test procedure, log reading worksheet)
• Project report PDF (IndiaAI and EKA-1 background, offline voice pipeline design, model-size honesty note, demonstration results, honest measurement limits)
• PPT presentation for final review
• Viva Q&A preparation document (why EKA-1 cannot run on a Pi, quantization and memory, Vosk vs language model roles, MQTT design, offline limits, language coverage)