Banking Support Intent Classification using NLP (Banking77)

Every bank's chatbot and support desk faces the same first problem: given a customer's message — 'I lost my debit card yesterday' — which of dozens of possible intents does it express? Get that routing wrong and the customer is sent down the wrong flow. Intent classification is the core NLP task behind chatbots, ticket triage and voice assistants, and Banking77 is the standard public benchmark: 13,083 real-style banking queries labeled across 77 fine-grained intents, from 'card lost' to 'interest rate enquiry'. This project fine-tunes DistilBERT on it — a transformer small enough to train on a single GPU yet strong enough to beat classical baselines convincingly — and evaluates with macro F1 across all 77 intents, not just headline accuracy. The interactive demo lets you type banking questions and watch the routing decision with its evidence.

1. The 13,083 Banking77 queries are loaded with their 77 intent labels and split into train/validation/test sets.
2. Queries are tokenized with the DistilBERT WordPiece tokenizer (max length 64) with padding and attention masks.
3. distilbert-base-uncased gets a classification head (dropout → dense 77 → softmax) and is fine-tuned with AdamW (lr 2e-5) for up to 5 epochs with early stopping on validation macro F1.
4. The held-out test set is evaluated once: accuracy, macro F1 and per-intent precision/recall are generated, plus a confusion analysis of the most-confused intent pairs.
5. In the web demo, your query is scored across the representative intent set and the top intent is shown with confidence bars and a suggested response. The demo's keyword scorer illustrates the output format; the shipped build runs the fine-tuned transformer.

Tech stack:

• Python 3, Hugging Face Transformers
• DistilBERT (distilbert-base-uncased), PyTorch
• scikit-learn (metrics, classification report)
• Jupyter Notebook (training & evaluation)
• HTML5 + JavaScript (interactive intent demo)
• Banking77 dataset (PolyAI, 13,083 queries)

Dataset & model details

• Dataset: Banking77 (PolyAI) — 13,083 online banking customer-service queries labeled across 77 fine-grained intents (e.g. card lost, balance enquiry, loan interest rate). The standard public benchmark for intent classification.
• Task: 77-class text classification; input = customer query (WordPiece tokens, max length 64), output = probability distribution over 77 intents.
• Model: DistilBERT (distilbert-base-uncased, 6 layers, 768 hidden dim) fine-tuned with a dropout → dense-77 → softmax head; AdamW (lr 2e-5), 5 epochs, batch 32, early stopping on validation macro F1.
• Metrics: Test accuracy ≈ 93%, macro F1 ≈ 0.92 (design targets for the built-to-order training run), per-intent precision/recall, confusion analysis. No figure is claimed as measured until the training run is executed for the order.

• [DistilBERT fine-tuning] distilbert-base-uncased fine-tuned on all 77 Banking77 intents — 6 transformer layers, 768-dim [CLS] representation, dropout → dense-77 → softmax, trained with AdamW and early stopping.
• [Interactive intent demo] Type any banking support question; the demo scores it across 12 representative intents with keyword evidence and returns the predicted intent, confidence bars and a suggested agent response.
• [Real benchmark dataset] Banking77 by PolyAI — 13,083 customer-service queries, 77 fine-grained intents — the public standard this task is actually evaluated on.
• [Macro-F1 evaluation] Per-intent precision/recall with macro F1 (≈ 0.92 design target) so rare intents count as much as common ones — the honest metric for 77-class problems.
• [Confusion analysis] The report examines which intent pairs the model confuses (e.g. card-related intents) — strong viva material.
• [Full training notebook] Tokenization (WordPiece, max length 64), dataset preparation, fine-tuning loop, evaluation and error analysis in one reproducible notebook.
• [Suggested-response mapping] Each intent maps to a draft agent response, showing how classification connects to a real support workflow.
• [Exported fine-tuned model] Saved transformer weights + tokenizer, so the demo runs the real model without retraining.

• Complete fine-tuning Jupyter notebook
• Fine-tuned DistilBERT weights + tokenizer
• Interactive intent-classification web demo with suggested responses
• Per-intent metrics, confusion analysis and training-curve plots
• Project report PDF (background, transformer theory, fine-tuning methodology, evaluation, error analysis)
• PPT presentation for final review
• Viva Q&A preparation document (transformers, attention, fine-tuning vs feature extraction, macro F1)

• The demo's browser scorer covers 12 representative intents with keyword evidence to illustrate the output format; the shipped build fine-tunes on all 77 intents.
• Queries far outside banking support (other domains, other languages) are out of scope — the model only knows its 77 intents.
• All performance figures are design targets for the training run, stated honestly — the report documents the actual achieved figures after training.
• Fine-tuning needs a GPU for reasonable training time; the notebook documents the expected setup.
• Suggested responses are draft templates, not approved bank communications.