Text Summarization using Transformers

News readers drown in articles they will never finish: readers face more long-form reporting than they can finish, and newsrooms need reliable auto-generated briefs and digests. Extractive methods — picking whole sentences — hit a ceiling decades ago because they cannot paraphrase, compress or fuse information across sentences. Transformer encoder-decoders changed this: BART, T5 and PEGASUS read the full article with self-attention and write a new, shorter text like a human sub-editor. The CNN/DailyMail corpus (Hermann et al., 2015; 287,113 training articles paired with journalist-written highlights) is the standard benchmark that made this progress measurable. This project fine-tunes those models with the Hugging Face Transformers library, evaluates with ROUGE-1/2/L on the held-out test split, plus the two analyses that make the report credible: a hallucination audit and an entailment-based factual-consistency check. The honest limitations (hallucination, the 1024-token window, position bias in the highlights) are documented, not hidden.

1. The CNN/DailyMail corpus is loaded (287,113 train / 13,368 validation / 11,490 test article-highlight pairs) and profiled for length distributions.
2. Articles are tokenized with the model tokenizer, truncated to 1024 input tokens; highlights become 128-token targets.
3. BART-large (and T5-base / PEGASUS-large in the comparison runs) is fine-tuned with AdamW, learning rate 3e-5 and label smoothing; validation ROUGE is tracked per epoch.
4. Summaries are generated on the test split with beam search (4 beams, length penalty 2.0) and scored with ROUGE-1/2/L.
5. A 50-article hallucination audit and an entailment-based factual-consistency check are run on sampled outputs and reported next to the ROUGE numbers.
6. The browser demo offers instant extractive summarization for interface testing, while the notebook holds the real transformer results.

Tech stack:

• Python 3.10, PyTorch, Hugging Face Transformers (BART, T5, PEGASUS)
• Hugging Face Datasets (cnn_dailymail loading and splits)
• rouge-score (ROUGE-1/2/L evaluation)
• NLTK (sentence tokenization for the demo and audit tooling)
• Jupyter notebook (buyer-run fine-tuning and evaluation)
• HTML/CSS/JS demo (live extractive summarizer)

• [Three-model fine-tuning] BART-large, T5-base and PEGASUS-large fine-tuned on CNN/DailyMail with the Hugging Face Transformers trainer
• [Full training recipe] Tokenization, 1024-token truncation, 128-token targets, AdamW, label smoothing, beam-search decoding with length penalty — all in the notebook
• [ROUGE evaluation] ROUGE-1, ROUGE-2 and ROUGE-L computed on the 11,490-article test split with the standard rouge-score package
• [Hallucination audit] Manual audit protocol over 50 sampled summaries, categorizing invented facts — the section examiners actually read
• [Factual-consistency check] Entailment-based scoring of summary against source article, reported alongside ROUGE
• [Long-document handling] Sliding-window chunking strategy for articles beyond the 1024-token input limit, with its trade-offs documented
• [Live browser demo] Extractive frequency-based summarizer with ROUGE-1 vs reference, compression stats and adjustable summary length — runs with no GPU
• [Decoding experiments] Beam size, length penalty and repetition-penalty ablations showing how decoding choices move the scores

• Complete source code (data loading, fine-tuning, decoding, ROUGE evaluation, audit tooling, demo)
• Jupyter training and evaluation notebook (buyer-run: fine-tunes and evaluates on your GPU)
• Project report PDF (background, transformer architecture, methodology, results, hallucination audit)
• PPT presentation for final review
• Viva Q&A preparation document (attention, BART vs T5 vs PEGASUS, ROUGE, beam search, hallucination)
• Setup guide (environment, GPU notes, dataset download, running fine-tuning and the demo)

• Abstractive models hallucinate — they invent facts absent from the source article; the report audits this on 50 samples instead of pretending ROUGE measures truth.
• The 1024-token input window truncates or chunks long documents; chunking loses cross-chunk context and the report documents the trade-off.
• CNN/DailyMail highlights are extractive-biased toward lead sentences, so fine-tuned models learn position bias — strong on news, weaker on other genres.
• ROUGE measures n-gram overlap, not factuality or readability; the report pairs it with the consistency check for that reason.
• Fine-tuning BART-large/PEGASUS needs a GPU with sufficient VRAM (16GB+ recommended); the setup guide gives realistic hardware guidance and a smaller-model fallback.
• The browser demo is extractive (frequency-ranked sentences), not a transformer — it demonstrates the interface, while the real abstractive results come from the notebook.