In this guide
How Offline Multilingual Voice Assistants Work: Wake Word, Speech, and Small Language Models
Most voice assistants only answer while they can reach a server. Speak to one in a hostel with patchy internet, or in a classroom where sending every spoken word to a cloud service is not acceptable, and it goes silent. An offline voice assistant solves that by doing the whole loop inside the room: it hears the words, understands them, and speaks the answer, all on a small computer that never sends audio anywhere. This guide explains how that loop works, stage by stage, why a giant model like EKA-1 cannot run on a Raspberry Pi, and what a small quantized model honestly can do instead.
The headline behind this guide arrived on 3 November 2026, when the IndiaAI Mission unveiled three indigenous foundation models in New Delhi: EKA-1, built by Soket Labs, with 26.3 billion parameters trained on about 7.5 trillion tokens and covering all 22 scheduled Indian languages plus English; and Dhir 2B and Dhir 5B, built by ZenteiQ AiTech, open-source models for English and 10 Indian languages, described as built for secure on-premise use where connectivity is limited. Those are the reported figures from the launch coverage. No independent benchmarks were part of the announcement, and the models themselves are not the subject of this guide. The subject is the engineering question the launch raises: how do you build a voice assistant that understands Indian languages and works with no internet at all?
The five stages, end to end
Every offline voice assistant, from a commercial smart speaker in airplane mode to a student's Raspberry Pi build, runs the same five stages in the same order. A microphone captures audio. A wake-word detector listens for one trigger phrase. A speech recognizer turns the spoken sentence into text. A language model decides what the sentence means and writes a reply. A speech synthesizer turns that reply back into audio, and a device-control step acts on commands. If any stage needs the internet, the assistant is not offline. A genuinely offline build keeps all five on the device.
The companion project for this guide is a Raspberry Pi 5 device that demonstrates exactly this loop in Hindi, Marathi, Tamil and English, with a dashboard showing the conversation log: EKA-1-Inspired Offline Multilingual Voice Assistant Project for Indian Languages. The rest of this guide explains the technology inside that build, so you can read the guide first and the project page second, or the other way around.
Stage 1: capturing the voice
Speech is air pressure changing over time, and a microphone turns those changes into numbers. A USB microphone array samples the sound tens of thousands of times per second and hands the Pi a stream of amplitude values. An array, rather than a single microphone, matters in a real room: several microphones spaced a few centimetres apart hear the same voice arriving at slightly different times, and the software can combine those signals to favour sound coming from the speaker's direction and suppress noise from elsewhere. That is beamforming, and it is the reason a voice assistant can hear a question from across a room while a laptop microphone cannot.
The practical limit at this stage is the room itself. Hard walls bounce sound, fans add a steady hum, and two people talking at once mix into one waveform the software cannot separate. An offline build does not fix any of that with more compute; it fixes it with a quiet room and a speaker standing close to the microphones. Any demo that claims otherwise is hiding the room conditions.
Stage 2: the wake word, a small pattern match
A voice assistant cannot run its full language model all day. The model is too large and too power-hungry, and listening continuously would mean recording everything said in the room. The wake word solves both problems. A tiny recognizer, a few megabytes in size, watches the audio stream for one short phrase, such as "Namaste Pi", and nothing else happens until it hears it.
A wake word is only a short audio-pattern match. The recognizer has learned what the phrase sounds like as a sequence of sound features, and it compares the incoming audio against that pattern many times a second. Because the pattern is short and fixed, the recognizer is cheap: it runs on a fraction of one processor core while the language model stays asleep. That is why the device can listen all day on a small computer, and why nothing is recorded or processed before the wake word. The trade-off is false triggers. A phrase that sounds similar, a television advert, or a cough can trip the pattern, so a good demo reports its false-trigger count honestly instead of claiming the detector never misfires.
Stage 3: speech recognition, on the device
Once woken, the assistant must turn speech into text. Offline speech recognition on a student build usually means Vosk, an open-source toolkit whose small models are about 40 MB per language. A Vosk model has two parts. The acoustic model maps sound features to the smallest units of speech, the phonemes. The language model, a different and much smaller thing than the large language model in stage 4, guesses which words those phonemes most plausibly form, using the statistics of the language.
Small models are the honest compromise of offline speech. A 40 MB model fits easily on a Pi and runs in real time, but it has less room for the rare words, accents and speaking styles that a large cloud model has seen. That is why the same sentence can be transcribed perfectly in Hindi and mangled in Marathi: the small models for different languages were trained on different amounts of data, and recognition quality follows the training data. A student demo should therefore test each language with the same scripted phrase set and report the misses per language, rather than claiming all four languages work equally well.
A related question students often ask is how Vosk compares with Whisper, the better-known open speech model. Whisper is more accurate, especially on noisy audio, but even its smallest useful models are several hundred megabytes and need far more compute per second of audio. On a Pi, Vosk's small models are the practical choice; Whisper belongs on a laptop or a server. Choosing between them is a trade of accuracy against the hardware you actually have.
Stage 4: the language model, and why size decides everything
The language model is the stage that turns a transcribed sentence into a reply. Given the words "light on", it must decide this is a command, not a question; given "what is the weather", it must decide this needs information it does not have and say so plainly instead of guessing. A small open-weight model of the 2B-parameter class does this by predicting the next word, one word at a time, using patterns it learned during training. It is not searching a database of answers. It is generating text that statistically follows the question.
The decisive question for any student build is whether the model fits in memory. A model's weights are numbers, and each parameter takes storage. In 16-bit floating point, one parameter takes 2 bytes, so a 2-billion-parameter model needs about 4 GB, and a 26.3-billion-parameter model like EKA-1 needs about 53 GB. Quantization shrinks those numbers by storing each weight in fewer bits: at 4-bit precision, roughly half a byte per parameter, a 2-billion-parameter model shrinks to about 1.1 GB, and EKA-1 shrinks to roughly 13 GB. The Pi 5 in this build has 8 GB of RAM, and the operating system, the speech models and the working space the model needs while it thinks all share that 8 GB. So a 2B-class quantized model fits comfortably, and EKA-1 does not fit at all, even quantized. That arithmetic is the whole reason the student build uses a small model in the same offline spirit as the Dhir models, rather than EKA-1 itself, and any project that claims otherwise has not done the memory maths.
Quantization has a cost. Rounding every weight to 4 bits loses information, so a quantized model gives slightly worse answers than the same model at full precision. The loss is small for short, simple replies, which is what a voice assistant produces, and large for long reasoning, which a voice assistant should not be doing anyway. The format matters too: the GGUF file format used by llama.cpp, the inference runtime in this build, stores the quantized weights plus the tables needed to run them on an ordinary CPU, with no graphics card. The runtime loads the file, reads the question, and generates the reply word by word, using the Pi's processor alone. Expect a reply in about two seconds for a short answer. That latency is a design target measured on the demo build, not a specification of the model.
Stage 5: speaking the answer, and doing something
The reply is text, and the assistant must speak it. Offline text-to-speech on a student build usually means Piper, a neural speech synthesizer that runs on a CPU with no internet. Piper turns text into sound in two conceptual steps: it predicts the sound features the sentence should have, including rhythm and stress, and then a vocoder turns those features into an audio waveform. Each voice is a separate file of roughly 21 to 137 MB, and Hindi is among the supported voices, which is why the build can answer in the language the question was asked in.
The second half of this stage is action. When the model decides the sentence is a command, the assistant sends a message to a local MQTT broker, a small message-relay program running on the Pi itself, which forwards it to a relay channel that switches a light or a fan. MQTT is a publish-and-subscribe protocol: the assistant publishes a message like "light on" to a named topic, and the relay, which subscribed to that topic, receives it and acts. Because the broker is local, the message never leaves the home network, and the command still works with the internet cable unplugged. That offline test, repeating a full exchange with no network, is the single most convincing demonstration in the whole build, and it is the one an examiner should ask to see.
Why Indian languages make this harder, and more interesting
English dominates the training data of almost every speech and language model, and Indian languages are under-represented by comparison. Three things follow. First, recognition quality varies by language, as stage 3 explained, because the small speech models were trained on different amounts of data. Second, people mix languages in a single sentence, saying "light on kar do" or "fan band karo", and a recognizer trained on pure Hindi or pure English stumbles on the mixed sentence. Handling code-mixing properly is research work, not a student project, and an honest build says so. Third, the written form matters: Hindi, Marathi and Tamil use different scripts, so the dashboard must render three scripts correctly and the speech models must be loaded per language rather than assumed to be one "Indian" model. The Dhir models and EKA-1 matter precisely because they were built for these languages; a student build that uses a small multilingual model inherits a little of that coverage and none of the scale.
Building the student version: scope, test, and honest measurement
A build that follows this guide has a fixed scope. The device is a Raspberry Pi 5 with 8 GB of RAM, a USB microphone array, a speaker, a small status display, and two relay channels. The software is Vosk with one small speech model per demo language, Piper with matching voices, llama.cpp running a 2B-class quantized model in GGUF form, a local MQTT broker, and a companion dashboard that shows the conversation log, the language picker, the wake-word status and the device switches. Everything is downloaded once during setup. After that, the build is offline.
Test it the way an examiner would. Prepare a scripted phrase set in each demo language: five questions and five commands per language. Run the set in a quiet room and record, per language, how many transcriptions were correct, how many replies were sensible, how many wake-word attempts were missed, and how long each reply took. Then unplug the network and run the set again to prove the offline claim. Then run it once with the television on, and record the difference. Those three runs, reported plainly with the misses included, are worth more than any accuracy figure copied from a model card, because they describe this build in this room.
The limits are part of the result. The assistant cannot give weather, news, live scores or any current facts without internet, and it must say so instead of guessing. The small model gives short, simple replies and misunderstands unusual phrasing. Recognition is weaker in Marathi and Tamil than in Hindi and English. The demo controls one light and one fan. Adding a language means adding a speech model and a voice, and adding a device means adding a relay and wiring. None of those limits is a failure; each one is a boundary the report should state.
Frequently asked questions
Can EKA-1 run on a Raspberry Pi?
No. EKA-1 has 26.3 billion parameters, so even in 4-bit quantized form it needs roughly 13 GB of memory before any working space, far beyond the Pi 5's 8 GB. The student build runs a 2B-class quantized open model of about 1.1 GB in the same offline spirit as the Dhir models. The difference is stated plainly in the project report, and the build is never presented as running EKA-1.
Why not just use a cloud voice assistant?
A cloud assistant is more accurate, knows more, and understands more languages. The offline build trades all of that for privacy and independence: no audio leaves the room, and the assistant works with no internet. Which trade is right depends on the use. A hostel room with patchy internet, a classroom that cannot send student voices to a server, and a home that values privacy are the cases where offline wins.
Is Vosk or Whisper the better speech recognizer?
Whisper is more accurate, particularly on noisy audio. Vosk's small models are the practical choice on a Pi because they are about 40 MB per language and run in real time on limited hardware, while Whisper's useful models are several hundred megabytes and need far more compute. On a laptop, the answer flips. The choice follows the hardware, not brand loyalty.
How long does a reply take?
About two seconds per short reply is the design target for the demo build, measured from the end of the spoken question to the start of the spoken answer. The time grows with longer replies, because the model generates one word at a time, and it grows sharply if the Pi is busy with other work. Measure it on your own build with a stopwatch and report the number you got.
Can I add another Indian language?
Yes, as an extension. Each new language needs a Vosk speech model and a matching Piper voice, both downloaded once, plus an entry in the dashboard language picker, and the new language is tested with the same scripted phrase set before it is shown as working. Recognition quality varies by language because the small speech models were trained on different amounts of data.
What should I read next?
Start with the companion project page, which is the build this guide explains: EKA-1-Inspired Offline Multilingual Voice Assistant Project for Indian Languages. Then read the related guides on edge deployment, quantization and MQTT linked below; together they cover the hardware, the model-size maths, and the messaging that holds the build together.