Voice Cloning Demo using Tortoise TTS

A few seconds of someone's voice is now enough for a neural model to speak arbitrary new sentences in a similar style — a capability with genuine uses in accessibility, personal assistants and creative media, and equally genuine misuse potential. Tortoise TTS made this capability open-source: it conditions synthesis on a short reference clip without any per-speaker training. This project turns that into a complete, honest student demo — reference upload with a quality check, a consent gate before synthesis, text-to-speech generation, and a comparison view that puts the original and cloned waveforms side by side with a computed similarity score. Nothing about quality is pre-claimed: cloning fidelity depends on the reference clip, and the comparison view exists precisely so the student can measure and discuss it. The ethics framing is part of the deliverable, not an afterthought.

1. The user uploads a 10–30 second reference clip; the quality checker verifies duration, clipping and noise level.
2. The user completes the consent declaration, which unlocks the synthesis stage and logs the declaration.
3. The reference clip is preprocessed and its voice characteristics are extracted as conditioning for Tortoise TTS.
4. Entered text is synthesized autoregressively in the reference voice style, then rendered to audio through the model's vocoder stage.
5. The comparison view aligns the original and cloned waveforms and computes an embedding cosine similarity for that specific run.
6. The result can be previewed in the browser and exported as a watermarked WAV file.

Tech stack:

• Python 3.10, PyTorch
• Tortoise TTS (open-source voice-cloning model)
• Torchaudio (audio preprocessing)
• Flask backend, HTML/CSS/JS demo frontend

• [Reference voice upload] Accepts a 10–30 second clean voice clip, with an automatic quality check for duration, clipping and noise level before synthesis is allowed.
• [Consent-gated workflow] Synthesis unlocks only after an explicit consent declaration for the reference voice; every output carries an inaudible watermark marker and the UI states the educational scope.
• [Text-to-cloned-speech synthesis] Tortoise TTS conditions generation on the reference clip's voice characteristics and renders new spoken text — no per-speaker training needed.
• [Similarity comparison view] Original and cloned clips are shown side by side with aligned waveforms, spectrogram-style views and a computed embedding cosine similarity for that run.
• [Consented sample voice library] Bundled sample voices are included so the demo runs instantly without uploading personal audio.
• [Voice settings] Sampling temperature and pacing controls let the student explore the quality-vs-stability tradeoff (configurable).
• [WAV export] Synthesized clips download as WAV files with the watermark marker documented in the report.

• Complete source code (preprocessing, synthesis pipeline, comparison view, web app)
• Consented sample voice library for instant demos
• Setup guide (environment, model weight download, GPU/CPU expectations)
• Project report PDF (background, Tortoise TTS architecture, methodology, ethics section, evaluation procedure)
• PPT presentation for final review
• Viva Q&A preparation document (TTS architectures, autoregressive synthesis, speaker conditioning, ethics of voice cloning)
• Demo guidance with a scripted walkthrough

• Cloning fidelity depends entirely on the reference clip — noisy, reverberant or very short clips produce noticeably weaker results.
• CPU synthesis is slow (several minutes per sentence); a CUDA GPU is strongly recommended for any live demo.
• The model is English-focused; other languages and heavy accents are outside its comfort zone.
• The similarity score is a computed embedding distance for that run, not a certified measure of "how good" the clone sounds — listening tests remain the real evaluation.
• This is an educational demo, not a production voice product: outputs are watermarked and the workflow requires consent.