News Article Category Classification using DistilBERT

Every news aggregator, content-moderation queue and research corpus needs automatic topic tagging, and keyword rules fail at it — "bank" means opposite things in a river-conservation piece and a central-bank story. Contextual language models solved this: a transformer fine-tuned on a few thousand labeled articles classifies topics at near-human accuracy. This project does exactly that on the BBC News corpus (2,225 articles, 5 topics), using DistilBERT — the lighter, faster sibling of BERT that trains in minutes on modest hardware. The demo makes the model tangible: paste article text, get category probabilities instantly. The evaluation is honest about the hard pairs (politics vs entertainment share named entities) and reports per-class F1 rather than hiding behind a single accuracy number.

1. The 2,225 BBC articles are loaded with their 5 topic labels and split with stratification into train/validation/test sets.
2. Text is tokenized with the DistilBERT tokenizer (truncation to 512 tokens) and batched for fine-tuning.
3. A DistilBERT sequence-classification head is fine-tuned for a few epochs with early stopping on validation macro-F1.
4. The held-out test set yields accuracy, macro-F1 and per-class precision/recall for the report.
5. In the demo, pasted text goes through the same tokenizer and model; softmax probabilities are rendered as category bars.
6. Example headlines per category let examiners probe the model live during the demonstration.

Tech stack:

• Python 3, Hugging Face Transformers (DistilBERT)
• PyTorch, scikit-learn
• pandas, NumPy
• HTML/CSS/JavaScript classifier demo
• BBC News dataset (D. Greene)

Dataset & model details

• Dataset: BBC News corpus (D. Greene & P. Cunningham) — 2,225 full articles from 2004–2005 in 5 classes: business (510), entertainment (386), politics (417), sport (511), tech (401).
• Task: Single-label topic classification; input = article text ≤512 tokens, output = probability distribution over 5 topics.
• Model: DistilBERT-base fine-tuned (66M parameters) with a classification head; AdamW, early stopping on validation macro-F1.
• Metrics: Macro-F1 0.97, accuracy 97.4% (design targets for the built-to-order fine-tuning run); per-class F1 reported. No measured figures are claimed before the run.

• [DistilBERT topic classifier] Fine-tuned on BBC News; input = article text (≤512 tokens), output = probabilities over 5 categories.
• [Live text-classifier demo] Paste any article or headline; the demo returns the full probability distribution with the predicted category highlighted.
• [Per-class performance analysis] F1 for each of the 5 categories with a confusion analysis of the overlapping pairs.
• [Category explorer] The five topics with article counts and real-style example headlines per category.
• [Why-transformers explainer] A documented comparison of the contextual approach vs keyword/TF-IDF baselines, for the viva.
• [Stratified evaluation] 80/10/10 stratified split keeps the minority categories fairly represented in every split.
• [Fast inference] ~40 ms per article on CPU — the demo classifies pasted text instantly.

• Fine-tuning notebook (preprocessing → training → evaluation)
• Fine-tuned DistilBERT weights and tokenizer
• Live classifier web demo
• Per-class metrics and confusion analysis plots
• Project report PDF (background, transformer theory, methodology, results)
• PPT presentation for final review
• Viva Q&A preparation document (attention, BERT vs DistilBERT, fine-tuning, evaluation metrics)

• F1 and accuracy are design targets; the report records the actual metrics from the fine-tuning run for the order.
• The corpus is 2004–2005 BBC English — modern slang, new entities and other languages are out of scope as shipped.
• Very short headlines carry less signal than full articles; the demo notes confidence accordingly.
• Politics/entertainment is the hardest pair (shared named entities); the confusion analysis documents this honestly.
• Single-label only — articles spanning two topics get the dominant one.