More project photos (3)
The problem
Humans read emotion from voice effortlessly — the tightened pitch of anger, the slowed energy of sadness — but machines need to learn those acoustic signatures from data. Speech emotion recognition (SER) matters wherever voice is the interface: call-center analytics, in-car assistants, mental-health screening tools and social robotics all need to know not just what was said, but how. The technical problem is a supervised audio-classification task: from a few seconds of speech, predict the speaker's emotional state. This project builds that classifier as a CNN–LSTM hybrid — the CNN reads local time–frequency patterns from mel spectrograms while the LSTM models how the emotion unfolds across the utterance — trained on RAVDESS, the Ryerson Audio-Visual Database of Emotional Speech and Song: 1,440 speech recordings from 24 professional actors (12 male, 12 female) across 8 emotions, with filename-coded labels that make the data pipeline transparent and auditable.
1,440 speech files from 24 actors, labeled with 8 emotions. Splits are speaker-aware so the same actor never spans train and test.
Follow the working sequence
How it works
- RAVDESS speech recordings (16-bit, 48 kHz .wav) are loaded and labels are parsed from the filename's emotion identifier — no manual annotation needed.
- librosa extracts per-utterance features: 40 MFCCs, 128-band mel spectrogram, chroma and zero-crossing rate, normalized per utterance.
- Data is split with speaker awareness so the same actor never appears in both train and test, preventing the model from memorizing voices instead of emotions.
- The CNN–LSTM is trained with class-weighted cross-entropy (neutral has half the samples — no "strong" intensity exists for it); accuracy and macro-F1 are logged per epoch with early stopping on validation F1.
- Evaluation computes per-class precision/recall, the confusion matrix and the classic calm↔neutral / happy↔surprised confusion analysis for the report.
- At inference, a new clip passes through the same feature pipeline and the softmax probabilities over the 8 emotions are displayed with the waveform and spectrogram.
Tech stack:
- Python 3.10, PyTorch or TensorFlow/Keras (CNN–LSTM)
- librosa (MFCC, mel spectrogram, chroma, ZCR extraction)
- scikit-learn (splits, confusion matrix, metrics)
- NumPy, pandas (feature tables)
- Jupyter notebook (buyer-run training and evaluation)
- Matplotlib, Seaborn (curves, confusion matrix)
- Flask audio-analysis demo UI
| Parameter | Value |
|---|---|
| Model | CNN (2 conv blocks) + BiLSTM (128 units) + dense softmax |
| Features | 40 MFCCs, 128-band log-mel, chroma, ZCR; per-utterance normalization |
| Dataset | RAVDESS: 1,440 speech files, 24 professional actors (12 M / 12 F) |
| Classes | 8 — neutral, calm, happy, sad, angry, fearful, disgust, surprised |
| Class balance | 192 samples per emotion except neutral (96); handled with class weights + stratified splits |
| Evaluation | Accuracy, macro-F1, confusion matrix on speaker-held-out test — computed by the notebook during your build |
| Input | Short speech clips (~3 s, 16-bit .wav) |
| Output | 8-class emotion probabilities with confidence bars |
Project features
- [40-dim MFCC features] Mel-frequency cepstral coefficients with 25 ms frames and 10 ms hop capture the timbral texture that carries emotional colour.
- [Mel spectrogram CNN input] 128-band log-mel spectrograms give the convolutional blocks a time–frequency image where angry speech shows dense high-energy bands and sad speech concentrates low.
- [Chroma + ZCR features] Pitch-class energy and zero-crossing rate add prosodic cues the cepstrum alone misses.
- [CNN–LSTM hybrid model] Two convolutional blocks extract local acoustic patterns; a bidirectional LSTM (128 units) models their evolution over the utterance; dense softmax outputs 8 emotion probabilities.
- [Waveform & spectrogram console] The demo UI renders each clip's waveform and mel spectrogram alongside the prediction, so the features behind a decision are visible.
- [Stratified training notebook] Speaker-aware train/validation splits, class-weighted loss for the neutral-class imbalance, per-epoch accuracy/F1 logging and a confusion matrix.
- [Clip test bench] RAVDESS-style filename-coded test clips (modality–emotion–intensity–actor identifiers) with one-click analysis and confidence bars.
- [Configurable emotion set] The 8-class head can be reduced (e.g. 4 core emotions) or retrained on an added corpus such as CREMA-D or TESS.
What is included
- Complete source code (feature extraction, model, training loop, demo console)
- Jupyter training and evaluation notebook (buyer-run: extract features, train, evaluate, plot confusion matrix)
- Trained CNN–LSTM weights from the reference training run
- Project report PDF (background, feature design, methodology, evaluation, error analysis)
- PPT presentation for final review
- Viva Q&A preparation document (MFCCs, mel scale, CNN–LSTM design, class imbalance, speaker-independent splits)
- Setup guide (environment, RAVDESS download from Zenodo, training, running the console)
Advantages
- Recognizes 8 emotions from speech (neutral, calm, happy, sad, angry, fearful, disgust, surprised) using a CNN-LSTM network on MFCC, mel spectrogram, chroma and zero-crossing-rate features.
- 40-dim MFCC features with 25 ms frames and 10 ms hop capturing timbral texture, plus 128-band log-mel spectrograms giving the convolutional blocks a time-frequency image.
- CNN-LSTM hybrid: two convolutional blocks extract local acoustic patterns, a bidirectional LSTM (128 units) models their evolution over the utterance, and a dense softmax outputs 8 emotion probabilities.
- Trained on the RAVDESS corpus: 1,440 speech recordings from 24 professional actors, with speaker-aware train/validation splits and class-weighted loss for the neutral-class imbalance.
- Waveform and spectrogram console rendering each clip's waveform and mel spectrogram alongside the prediction, so the features behind a decision are visible.
- Stratified training notebook with per-epoch accuracy/F1 logging and a confusion matrix, plus a clip test bench with RAVDESS-style filename-coded test clips and confidence bars.
Disadvantages
- RAVDESS emotions are acted by professional actors reading two fixed sentences: acted speech is cleaner and more exaggerated than spontaneous emotion, so real-world accuracy will be lower, as the report states openly.
- Speaker-sensitive despite speaker-aware splits: unseen voices, accents and recording conditions degrade performance.
- Background noise, overlapping speech and low-quality microphones hurt the features; the build assumes single-speaker, reasonably clean clips.
- Emotion is predicted per whole utterance, so mixed or shifting emotions within one clip are not modelled.
- The corpus is North American English; other languages and heavy accents are outside the training distribution.
Limitations & prerequisites
- RAVDESS emotions are acted by professional actors reading two fixed sentences — acted speech is cleaner and more exaggerated than spontaneous emotion, so real-world accuracy will be lower; the report states this openly.
- The model is speaker-sensitive despite speaker-aware splits: unseen voices, accents and recording conditions degrade performance.
- Background noise, overlapping speech and low-quality microphones hurt the features; the build assumes single-speaker, reasonably clean clips.
- Emotion is predicted per whole utterance — mixed or shifting emotions within one clip are not modelled.
- The corpus is North American English; other languages and heavy accents are outside the training distribution.
Frequently Asked Questions
Which dataset is used?
RAVDESS (Ryerson Audio-Visual Database of Emotional Speech and Song, Livingstone & Russo 2018): 1,440 speech recordings from 24 professional actors covering 8 emotions, freely downloadable from Zenodo under CC BY-NC 4.0. Labels are encoded in the filenames.
Which model is used?
A CNN–LSTM hybrid: convolutional blocks read local patterns from mel spectrograms/MFCCs, a bidirectional LSTM models temporal evolution, and a dense softmax outputs the 8 emotion classes.
Is the accuracy guaranteed?
No fixed accuracy is claimed. The notebook computes accuracy, macro-F1 and the confusion matrix on your own speaker-held-out test split; the report presents those numbers with an honest discussion of acted-vs-spontaneous speech.
Is this project suitable for a final-year project?
Yes — for Computer Science, AI/ML and Data Science programs. It demonstrates audio signal processing, feature engineering, deep sequence modelling and rigorous evaluation with class-imbalance handling.
Can it work on live microphone input?
The feature pipeline accepts any .wav clip, so a live-record mode is a supported customization; the base build analyses uploaded/recorded clips through the console.
Can new emotions or datasets be added?
Yes — the classifier head is configurable, and merging CREMA-D or TESS recordings is documented as an extension, with relabelling notes for mismatched emotion sets. Suitable for B.E./B.Tech final-year projects in Computer Science, AI/ML and Data Science.
Components & software requirements
- Python 3.10, PyTorch or TensorFlow/Keras (CNN–LSTM)
- librosa (MFCC, mel spectrogram, chroma, ZCR extraction)
- scikit-learn (splits, confusion matrix, metrics)
- NumPy, pandas (feature tables)
- Jupyter notebook (buyer-run training and evaluation)
- Matplotlib, Seaborn (curves, confusion matrix)
- Flask audio-analysis demo UI
Delivery information
Built-to-order project. Delivery timeline is shared after order confirmation based on current queue.
Support terms
Complete documentation, setup guide, and viva preparation included. Support for setup and explanation provided.
Download free synopsis (PDF)
Enter your name and WhatsApp number — the synopsis PDF downloads immediately. We may send occasional project updates on WhatsApp; no spam.