News Article Summarization using T5 Transformer (CNN/DailyMail Dataset)

News readers face a volume problem: hundreds of long articles appear daily, and few readers get past the headline. Newsrooms have always handled this with hand-written highlights, but producing them is manual and inconsistent at scale. Automatic summarization promises a highlight for every article, instantly. The catch is that simple extractive methods can only copy existing sentences, so their summaries read disjointed and miss points spread across paragraphs. This project goes the full deep-learning route: the T5-base transformer is fine-tuned on the CNN/DailyMail corpus of over 300,000 articles paired with author-written highlights, learning to read an article and rewrite its essence in its own words. Generation quality is judged with ROUGE, the standard n-gram-overlap metric of summarization research, against an extractive baseline and a lead-sentences heuristic so the transformer's contribution is measured, not assumed.

1. Articles and highlights are loaded from Hugging Face (cnn_dailymail v3.0.0: 287,113 train, 13,368 validation, 11,490 test), cleaned of bylines and boilerplate, and truncated to 512 input tokens.
2. Each article is prefixed with "summarize:" and tokenized with the T5 SentencePiece tokenizer (32k vocabulary), framing summarization as T5's native text-to-text task.
3. T5-base is fine-tuned with teacher-forced cross-entropy, the AdamW optimizer, a learning rate near 5e-5 and 3–5 epochs on a single GPU (design targets), with mixed precision and gradient accumulation.
4. Checkpoints are selected by validation ROUGE-L; generation uses beam search (4 beams, length penalty tuned on validation) capped at 150 output tokens.
5. Final ROUGE-1/2/L F1 scores are computed once on the held-out 11,490-article test split, alongside the extractive and lead-sentence baselines run on the same split.
6. In the web demo, pasted articles are summarized through the trained model (the bundled instant demo runs the extractive baseline in-browser), and ROUGE against editorial highlights is computed live.

Tech stack:

• Python 3, PyTorch, Hugging Face Transformers
• T5-base (220M parameters), SentencePiece tokenizer
• cnn_dailymail v3.0.0 (Hugging Face Datasets)
• rouge-score (Google Research), NLTK
• Jupyter Notebook (training & evaluation)
• Single-file HTML/CSS/JavaScript web demo
• Flask inference wrapper (model API)
• Git

Dataset & model details

• Dataset: CNN/DailyMail news summarization corpus (Hermann et al., 2015), Hugging Face cnn_dailymail v3.0.0 — 287,113 training, 13,368 validation and 11,490 test article–highlight pairs; mean 781 article tokens and 56 highlight tokens; fields id, article, highlights.
• Task: Abstractive summarization; input = full news article (512 tokens max, design target) prefixed "summarize:", output = generated highlights-style summary (150 tokens max, design target).
• Model: T5-base encoder–decoder transformer (Raffel et al., JMLR 2020) — 12 encoder + 12 decoder blocks, approximately 220,000,000 parameters; trained with teacher-forced cross-entropy, AdamW, learning rate near 5e-5, 3–5 epochs (design targets).
• Metrics: ROUGE-1/ROUGE-2/ROUGE-L F1 (Lin, 2004) against reference highlights. Design targets for the buyer's training run: ROUGE-1 ≈ 0.42, ROUGE-2 ≈ 0.20, ROUGE-L ≈ 0.40. No score is claimed as measured until the training run executes.

• [Fine-tuned T5-base abstractive summarizer] T5-base (220M parameters) trained on 287,113 CNN/DailyMail articles to generate highlights-style summaries in its own words — genuine abstractive NLP, not sentence extraction.
• [Interactive web demo] Paste any news article and get a summary with word counts and compression ratio; four bundled news-style samples included for instant tries.
• [Live in-browser ROUGE workbench] The demo computes ROUGE-1/2/L F1 live in JavaScript against editorial highlights for each sample, shown next to the fine-tuned model's design targets.
• [Honest baseline comparisons] An extractive word-frequency summarizer and a lead-sentences heuristic are implemented so the report's comparison table measures what the transformer actually adds.
• [Complete training notebook] Data loading from Hugging Face, SentencePiece tokenization with the "summarize:" prefix, the fine-tuning loop, checkpointing and evaluation in one reproducible notebook.
• [Evaluation and failure analysis] ROUGE curves, sample input/output pairs and a 200-output hallucination spot-check, reported honestly rather than cherry-picked.
• [Inference wrapper] The trained weights served behind a simple API with exposed generation settings (beams, length penalty, max tokens), wired to the web demo.

• Complete T5 fine-tuning and evaluation Jupyter notebook
• Trained T5-base weights and tokenizer files
• Web demo (single-file app) with inference wrapper wired to the trained model
• ROUGE evaluation report with baseline comparison table and failure analysis
• Project report PDF (transformer background, methodology, results, hallucination audit)
• PPT presentation for final review
• Viva Q&A preparation document (transformers, attention, fine-tuning, ROUGE, baselines)