Speaker Identification using MFCC

"Who is speaking?" is one of the oldest problems in speech technology — behind voice-based access control, meeting transcription ("who said what") and forensic audio analysis. Unlike speech recognition (what was said), speaker identification is text-independent: it must recognize a voice regardless of the words spoken. The acoustic basis is the vocal tract: each speaker's physiology shapes the spectral envelope of their speech in a stable, learnable way, and Mel-Frequency Cepstral Coefficients (MFCCs) are the classic compact representation of that envelope. The standard academic pipeline — MFCC features, a Universal Background Model (GMM-UBM) adapted per speaker, and similarity scoring — is still the clearest way to learn the whole field, and it scales naturally to modern embedding (x-vector) scoring. This project implements that pipeline end to end with an enrollment console, an MFCC heatmap visualizer and closed-set identification plus 1:1 verification trials.

1. Speech is framed into 25 ms windows with 10 ms shift; each frame is Hamming-windowed, passed through a mel filterbank, log-compressed and DCT-transformed into 40 MFCCs, plus first and second deltas.
2. Voice activity detection trims silence; per-utterance cepstral mean/variance normalization removes channel bias.
3. A 512-mixture GMM Universal Background Model is trained on pooled multi-speaker speech to represent speaker-independent acoustic space.
4. Each enrolled speaker's model is MAP-adapted from the UBM using their enrollment utterances (~30 s of speech recommended per speaker).
5. At test time, the utterance's frames are scored against every enrolled speaker model (log-likelihood ratio vs the UBM); the embedding path independently ranks by cosine similarity.
6. The speaker with the best combined score above the rejection threshold is returned with ranked alternatives; below threshold, the voice is rejected as unknown.

Tech stack:

• Python 3.10, scikit-learn (GMM-UBM, MAP adaptation)
• librosa (MFCC extraction, VAD)
• NumPy, SciPy (scoring, likelihood ratios)
• PyTorch (embedding/TDNN path)
• Jupyter notebook (buyer-run training and evaluation)
• Matplotlib (MFCC heatmaps, DET curves)
• Flask identification console UI

• [40-dim MFCC + deltas] 25 ms frames with 10 ms shift, Hamming window, 40 cepstral coefficients plus Δ and ΔΔ — 120-dim frames capturing the vocal-tract spectral envelope.
• [GMM-UBM speaker models] A 512-mixture Universal Background Model trained on pooled speech, MAP-adapted per enrolled speaker — the classical text-independent identification recipe.
• [Embedding cosine scorer] A TDNN-style embedding path projects utterances to fixed vectors scored by cosine similarity, shown alongside GMM log-likelihood-ratio scores.
• [Enrollment console] Add speakers with their utterances, inspect per-speaker models, and re-enroll — the closed-set gallery the identifier searches.
• [MFCC heatmap visualizer] Renders the 40×time cepstral matrix of any test utterance so the features behind a decision are inspectable.
• [Identification + verification] 1:N ranked identification with similarity bars, plus 1:1 accept/reject verification trials against a tuned threshold.
• [Evaluation notebook] Held-out trial scoring with identification accuracy, DET-style analysis and threshold tuning — every number comes from your build.
• [Channel-matched protocol] Enrollment and test guidance keeps microphone/channel conditions matched, with the degradation documented when they are not.

• Complete source code (MFCC pipeline, GMM-UBM training, MAP adaptation, scorers, console UI)
• Jupyter training and evaluation notebook (buyer-run: extract features, train UBM, enroll, score trials)
• Reference speaker models and the enrollment console
• Project report PDF (background, feature theory, methodology, evaluation, error analysis)
• PPT presentation for final review
• Viva Q&A preparation document (MFCC computation, GMM-UBM, MAP adaptation, identification vs verification, channel effects)
• Setup guide (environment, enrolling speakers, running identification trials)

• This is closed-set identification: it picks the best match among enrolled speakers and can only reject unknowns via threshold — it cannot name someone never enrolled.
• Channel mismatch is the classic failure mode: enrollment on a studio mic and testing on a phone degrades scores sharply; the protocol requires matched conditions and the report documents why.
• Short utterances (under ~3 s) carry little speaker information and score unreliably; overlapping speakers and heavy background noise are outside the build's scope.
• Accuracy depends entirely on the enrollment data you provide: few or noisy enrollment utterances produce weak speaker models.
• This is an academic prototype, not a security product — voice biometrics for access control need anti-spoofing the build does not include.