Handwritten Text Recognition using CRNN

Handwriting remains one of the hardest problems in optical character recognition: every writer's slant, stroke width and letter joins differ, which is why off-the-shelf print OCR collapses on cursive notes and answer scripts. Digitizing handwritten pages — notes, forms, exam scripts, historical documents — currently means slow, expensive manual re-typing, because generic OCR tools were built for printed text and fail on variable-width handwritten lines. The core difficulty is sequence modeling: characters are not neatly boxed, so the model must learn where one letter ends and the next begins while reading each line left to right. A CRNN addresses exactly this — a CNN extracts visual features, a bidirectional LSTM reads the sequence with full line context, and CTC decoding produces the character string without needing character-level alignment labels. For students, this is a complete deep-learning pipeline with measurable outcomes: line segmentation, sequence decoding and error-rate metrics that are computed, not claimed.

Frequently asked questions

1. What is a CRNN and why is it used for handwriting? A Convolutional Recurrent Neural Network combines a CNN (visual feature extraction) with a BiLSTM (sequence context) and CTC decoding, which converts frame predictions into text without needing character-level alignment labels — ideal for variable-width handwritten lines.
2. What dataset is the model trained on? The IAM Handwriting Database — 13,353 labeled handwritten English text lines — widely used as the standard benchmark for offline handwriting recognition research.
3. How is the model's performance evaluated? Character Error Rate (CER) and Word Error Rate (WER), computed by the training notebook on the validation split using edit distance at character and word level.
4. Will it read my own handwriting in the demo? It performs best on neat print-style handwriting similar to the corpus; augmentation improves robustness, but highly idiosyncratic cursive remains a known limitation.
5. Is this project suitable for a final-year project? Yes. It suits B.E./B.Tech students in Computer Science, AI/ML and Data Science, demonstrating sequence modeling, CTC loss, CER/WER evaluation, data augmentation and a live transcription demo — strong, explainable viva material.
6. What will I receive? Complete source code, trained .pth checkpoints, training and evaluation notebooks, line-segmentation and preprocessing utilities, project report PDF, PPT presentation, viva Q&A document and a setup guide.

Dataset & model:
Dataset name: IAM Handwriting Database.
Source: publicly released academic benchmark for offline handwriting recognition.
Task: line-level handwritten text recognition (sequence prediction).
Classes: character vocabulary of English letters, digits and common punctuation.
Model: CRNN — CNN encoder plus bidirectional LSTM, decoded with CTC; trained with the CTC loss in PyTorch.
Input: scanned line image, normalized to fixed height (full pages are segmented into lines first).
Prediction: frame-level character probabilities decoded into a character sequence.
Output: transcribed text per line with per-line confidence scores; exportable as plain text or CSV.
Evaluation metrics: Character Error Rate (CER), Word Error Rate (WER), CTC loss curves, confusion-style error analysis — computed by the training notebook on the validation split during the build.
Design targets: <10% CER and <30% WER on the IAM validation split. These are design targets, not measured claims; the notebook computes the actual rates during the build.

Working:

1. Training phase: IAM text-line images are augmented (slant, stretch, noise) for writer-style robustness and fed to the CRNN; the network trains with the CTC loss, and the notebook logs CTC loss curves on the validation split.
2. Evaluation phase: the training notebook decodes validation lines and computes Character Error Rate and Word Error Rate, plus error-analysis plots of commonly confused characters.
3. Inference phase: the user uploads a scanned page; the preprocessing pipeline deskews, binarizes and normalizes it.
4. The line-segmentation utility splits the page into individual text-line images.
5. Each line image is normalized to a fixed height, passed through the CNN encoder to produce a feature sequence, and read left-to-right and right-to-left by the bidirectional LSTM.
6. CTC decoding converts frame-level predictions into the character sequence without alignment labels; the web app displays the transcription with per-line confidence scores.
7. Transcriptions can be exported as plain text or CSV for downstream use.

Specifications:
Model | CRNN (CNN encoder + BiLSTM + CTC decoding)
Dataset | IAM Handwriting Database (13,353 handwritten text lines, line-level task)
Design targets | <10% CER, <30% WER on the validation split (design targets, not measured claims)
Input | Scanned line image, normalized to fixed height
Vocabulary | English letters, digits and common punctuation
Inference | CPU-friendly, under 1 s per line
Platform | Windows/Linux/macOS, Python 3.10; web demo included

[Line-Level Transcription] (implemented) — Transcribes handwritten English text from scanned or photographed pages, line by line, through the web demo.
[CRNN Architecture] (implemented) — CNN feature extractor plus bidirectional LSTM with CTC decoding, the standard approach for sequence-like text recognition without alignment labels.
[Per-Line Confidence Scores] (implemented) — The demo shows each transcribed line with its confidence, so uncertain lines are visible rather than silently wrong.
[CER/WER Evaluation] (implemented) — The training notebook computes Character Error Rate and Word Error Rate on the validation split, with error-analysis plots of commonly confused characters.
[Line-Segmentation Utility] (implemented) — Automatically splits full scanned pages into individual text-line images before transcription.
[Writer-Style Augmentation] (implemented) — Slant, stretch and noise augmentation during training improve robustness across handwriting styles.
[Export Options] (implemented) — Transcriptions can be exported as plain text or CSV for downstream use.
[CPU-Friendly Inference] (implemented) — The lightweight model transcribes in under a second per line on a regular laptop CPU; no GPU needed for the demo.

Complete source code (training, evaluation, inference, web app)
Trained CRNN weights (.pth checkpoints)
Training and evaluation notebooks (CER/WER computation, CTC loss curves, error analysis)
Line-segmentation and preprocessing utilities
Project report PDF (background, CRNN methodology, dataset analysis, evaluation)
PPT presentation for the final review
Viva Q&A preparation document (CRNN, CTC loss, CER/WER metrics, sequence modeling)
Setup guide (environment, dependencies, dataset download steps)

Cursive and heavily slanted handwriting lowers transcription accuracy — the model is trained on the IAM corpus writing styles.
The IAM database is English-only; other scripts are out of scope.
Needs clean line segmentation: overlapping lines or dark ruled paper can confuse the segmenter.
Requires legible scans; very low-resolution or blurry images degrade transcription quality.
All reported figures are design targets from the buyer's own training run; no pre-measured CER/WER is claimed.