Chatbot using Seq2Seq

Most chatbots students meet are rule-based: pattern–response pairs that break the moment a user phrases something unexpectedly. The alternative is to learn conversation from data, and the sequence-to-sequence (seq2seq) encoder–decoder is the architecture that made that practical — the same idea behind the first generation of neural chat systems and neural machine translation. The problem it addresses is direct: given a user utterance of arbitrary wording, produce a fluent, contextually plausible response without hand-written rules. Training such a model is a genuinely instructive deep-learning exercise — tokenization and vocabulary construction, variable-length sequence batching, teacher forcing, attention, perplexity-based evaluation — each a viva-ready concept. This project implements the full pipeline on the Cornell Movie Dialogs corpus, a metadata-rich collection of 220,579 fictional conversational exchanges between 10,292 pairs of movie characters (304,713 utterances, 617 movies), filtered to ~96,000 prompt–response pairs for training.

1. The Cornell Movie Dialogs corpus is parsed into consecutive prompt–response utterance pairs, filtered for length and cleaned of stage directions and encoding artifacts.
2. Text is tokenized and mapped to integer sequences using a 20,000-word vocabulary; [start] and [end] tokens bracket every target reply, and batches are padded to the longest sequence.
3. The bidirectional GRU encoder reads the prompt and returns a hidden state per input token; the decoder is initialized from the encoder's final state.
4. During training, the decoder predicts each reply token conditioned on the previous ground-truth token (teacher forcing, annealed 1.0 → 0.5) and the attention-weighted encoder context; cross-entropy loss is backpropagated with Adam.
5. Every epoch logs train and validation loss plus validation perplexity; the best-perplexity checkpoint is kept and training stops early if perplexity plateaus.
6. At inference, the chat UI feeds the user's message through the encoder, then the decoder generates the reply autoregressively — greedily or via beam search — until [end] or the length cap.

Tech stack:

• Python 3.10, PyTorch (GRU encoder–decoder, Bahdanau attention)
• NLTK / regex tokenizer, custom vocabulary builder
• NumPy (batching, padding utilities)
• Jupyter notebook (buyer-run training and evaluation)
• Matplotlib (loss/perplexity curves, attention heatmaps)
• Flask chat demo UI

• [GRU encoder–decoder] A 2-layer bidirectional GRU encoder (512 hidden units) compresses each prompt into hidden states; a 2-layer GRU decoder generates the reply token by token from the encoder context.
• [Bahdanau attention] Additive attention lets the decoder weight every encoder position at each generation step, so longer prompts keep their meaning instead of collapsing into one fixed vector.
• [Attention heatmap visualizer] For any prompt–reply pair, the demo renders the α(t,i) attention matrix so you can see exactly which input words the decoder focused on while generating each output word.
• [Beam-search decoding] Inference supports greedy decoding and beam search (width 5), with ranked hypotheses and log-probabilities shown for comparison.
• [Dialogue preprocessing pipeline] Corpus cleaning, prompt–response pairing, lowercasing, punctuation handling, vocabulary construction (20,000 most frequent tokens) and length filtering (≤20 tokens) in one reproducible script.
• [Training notebook with live metrics] Logs train/validation loss and perplexity per epoch, teacher-forcing schedule, and early stopping on validation perplexity — every number in the report comes from your own run.
• [Interactive chat demo] A clean chat UI with typing indicator, token counts and suggestion prompts, served from the trained weights for live demonstrations.
• [Configurable decoding] Temperature, beam width and maximum reply length are adjustable at inference time for experimentation.

• Complete source code (preprocessing, model, training loop, attention visualizer, chat demo)
• Jupyter training and evaluation notebook (buyer-run: preprocess, train, log perplexity, visualize attention)
• Trained seq2seq weights (.pt) from the reference training run
• Project report PDF (background, architecture, training procedure, evaluation, error analysis)
• PPT presentation for final review
• Viva Q&A preparation document (encoder–decoder, attention math, teacher forcing, perplexity, decoding strategies)
• Setup guide (environment, corpus download, training on CPU/GPU, running the chat demo)

• Seq2seq models trained with cross-entropy favour safe, generic replies ("I don't know", "That's interesting") — the report discusses this known behaviour and how beam search only partly mitigates it.
• The model has no memory across turns: each reply is generated from the single current prompt, so multi-turn context and pronoun resolution are limited.
• Prompts and replies are capped at 20 tokens; longer inputs are truncated and may lose meaning.
• The dialogue style mirrors movie scripts — dramatic and informal — which does not transfer to formal or domain-specific conversation.
• Training the full configuration takes several hours on a GPU and considerably longer on CPU; the notebook documents expected runtimes per hardware class.