Deepfake Voice Detection Using Mel-Spectrograms and Machine Learning

Cloned voices are now good enough to fool human ears — the same voice-cloning tools that dub videos convincingly are also used in impersonation scams, which is why detecting synthetic speech has become a serious research problem. A human listener cannot reliably tell a neural-vocoder voice from a real one, and hand-written acoustic rules break every time a new synthesis algorithm appears. This project takes the data-driven route used by the actual research community: convert speech into mel-spectrograms and train a convolutional neural network to spot the machine traces that synthesis leaves behind. The training and evaluation follow the ASVspoof 2019 Logical Access challenge protocol, so every result is measured against a public benchmark with official baselines instead of a private test set. The web demo makes the pipeline tangible — load a voice sample and watch the app compute a real 64-band mel-spectrogram and score it. Note that this is an AUDIO project (spoofed speech), completely different from deepfake image detection.

1. The dataset is loaded: 2,580 bonafide and 22,800 spoofed utterances for training (attacks A01–A06: four TTS and two voice-conversion algorithms), with disjoint speakers held out for development and evaluation.
2. Every utterance is resampled to 16 kHz and framed with a 25 ms window and 10 ms hop; a 512-point FFT feeds 64 triangular mel filters spanning 80–8000 Hz, and the log energies form the 64 × 278 input for a 2.8 s clip.
3. The compact CNN trains with binary cross-entropy and Adam, learning the texture differences between genuine and synthesized spectrograms — vocoder banding, phase flicker and high-band tonal artifacts.
4. The development set (same six attacks, new speakers) is used for early stopping and threshold selection; the model never sees the evaluation attacks during training.
5. Final evaluation runs on the held-out set — 7,355 bonafide and 63,882 spoofed utterances including 11 attack algorithms unseen in training — reporting equal error rate and the min t-DCF cost.
6. For inference, any new recording passes through the identical front end and the trained network outputs a bonafide-vs-spoof probability with the calibrated decision threshold.
7. The web demo mirrors this pipeline in the browser: it parses audio, computes the same 64-band log-mel spectrogram (measured at approximately 60–240 ms per 2.8 s clip in the demo build, varying with device load) and scores it with a handcrafted artifact heuristic, since model weights cannot ship inside a zero-dependency demo page.

Tech stack:

• Python 3, PyTorch
• ASVspoof 2019 LA dataset
• Mel-spectrogram front end (25 ms / 10 ms, 64 bands)
• Compact CNN (conv blocks · global average pooling · sigmoid)
• NumPy · SciPy (signal processing)
• Matplotlib (DET curves, training plots)
• Jupyter Notebook (training & evaluation)
• HTML5 · CSS3 · JavaScript (single-file web demo)

Dataset & model details

• Dataset: ASVspoof 2019 Logical Access — the public benchmark for synthetic-speech detection from the ASVspoof challenge (asvspoof.org; official release via the University of Edinburgh DataShare, registration required). Train: 2,580 bonafide + 22,800 spoofed utterances (attacks A01–A06: 4 TTS + 2 voice conversion, 20 speakers). Development: 2,548 bonafide + 22,296 spoofed (same attacks, disjoint speakers). Evaluation: 7,355 bonafide + 63,882 spoofed (attacks A07–A19, of which 11 are unseen in training).
• Task: Binary classification (presentation-attack detection for voice). Input = 64 × 278 log-mel spectrogram of a ~2.8 s utterance at 16 kHz. Output = probability of the utterance being spoofed (TTS or voice conversion).
• Model: Compact CNN (design target ≈ 0.4 M parameters): three 3×3 convolution blocks with batch normalization, ReLU and pooling → global average pooling → 128-d embedding → dense layer with sigmoid. Loss: binary cross-entropy. Optimizer: Adam.
• Metrics: Equal error rate (EER) and min t-DCF on the evaluation set. Official challenge baselines to beat: LFCC-GMM 8.09% EER / 0.2116 min t-DCF and CQCC-GMM 9.57% / 0.2366 (reported in the ASVspoof 2019 evaluation summary, Wang et al.). Design target: EER under 5% on the evaluation set — a target, not a measured claim; the report documents the actual figure after the training run. No accuracy is claimed as measured until training is executed for the order.

• [Mel-spectrogram front end] Raw audio is resampled to 16 kHz and converted to 64-band log-mel spectrograms (25 ms window, 10 ms hop, 80–8000 Hz) — the same front end the anti-spoofing literature uses, implemented as a documented preprocessing module.
• [Compact CNN classifier] Three convolution blocks (3×3 kernels, batch normalization, ReLU, pooling) followed by global average pooling and a dense sigmoid head — approximately 0.4 M parameters by design, small enough to run inference on a laptop CPU.
• [ASVspoof 2019 LA training pipeline] A complete training notebook: dataset loading, train/dev splits with disjoint speakers, augmentation, binary cross-entropy training with Adam, and checkpointing on development-set EER.
• [EER evaluation harness] The official challenge metric — equal error rate plus DET-curve plotting — computed on the held-out evaluation set with its 11 unseen attack algorithms, so the report shows honest generalization numbers.
• [Interactive web demo] A single-file HTML app that computes a real mel-spectrogram in the browser (FFT + mel filterbank in pure JavaScript) and visualizes it with waveform, feature bars and a detection verdict — no server, runs fully offline.
• [Upload and microphone input] The demo accepts WAV/MP3 uploads and 3-second microphone recordings; any real voice runs through the same analysis pipeline as the built-in samples.
• [Cross-dataset testing on WaveFake (optional)] Evaluating the trained model on the WaveFake neural-vocoder corpus can be added on request, to demonstrate how the detector behaves on vocoders outside the training distribution.
• [Streaming real-time detection (future scope)] Chunk-wise inference on a live audio stream is designed but not implemented; the report documents the buffering and latency considerations.

• Complete training & evaluation Jupyter notebook (preprocessing, CNN definition, training loop, EER/DET-curve evaluation)
• Trained CNN weights plus the mel-spectrogram preprocessing module
• Interactive single-file web demo (real in-browser spectrogram computation, upload + microphone input, two built-in samples)
• Project report PDF (voice-spoofing background, mel-scale theory, ASVspoof protocol, architecture rationale, methodology, results)
• PPT presentation for final review
• Viva Q&A preparation document (mel filterbanks, CNN design, EER vs accuracy, seen vs unseen attacks, vocoder artifacts)
• Dataset-setup guide (DataShare registration, directory layout, protocol file usage)
• Build timeline & support: preparation is confirmed at quotation (expected 2–3 weeks for the training run and full viva kit); build guidance over WhatsApp during preparation and demo support before the review.