Handwritten Equation Solver using CNN

Point-and-shoot equation solving is a classic "could a machine do it" demo that hides a serious pipeline: segmentation, symbol recognition, two-dimensional parsing and symbolic computation. A photo of "3x + 2 = 11" is trivial for a person and hard for software, because the symbols must first be cut apart, each one classified, their spatial relationships (superscript, fraction bar, subscript) recovered, and only then the mathematics evaluated. This project builds that full chain: a CNN trained on public handwritten-symbol data classifies every segment, a baseline-structure parser turns the classified symbols into an expression tree, and SymPy evaluates, simplifies or solves the resulting expression exactly.

The recognizer is trained on EMNIST (Cohen et al., 2017) for digits and letters plus CROHME competition data (ICDAR 2011-2019) for math operators and Greek letters, covering roughly 100 symbol classes. The included Jupyter notebook runs a symbol-accuracy procedure on a held-out symbol set and logs the solve rate on a curated equation sheet during your own build, so the report's numbers come from your run rather than a claim.

1. The equation photo is binarized and deskewed with OpenCV, then split into connected components.
2. Each component is normalized to the classifier's input size and classified by the CNN into a digit, letter or operator with a confidence score.
3. Spatial relationships between symbols are analyzed with recursive baseline-structure parsing to recover fractions, superscripts and roots.
4. The parser assembles the symbols and relationships into an expression tree.
5. The tree is converted to a SymPy expression and evaluated, simplified or solved symbolically.
6. The evaluation notebook runs the held-out symbol-accuracy procedure and logs solve-rate over the equation sheet.

Tech stack:

• Python 3.10, PyTorch (CNN training and inference)
• EMNIST dataset (Cohen et al. 2017, digits and letters)
• CROHME competition symbols (ICDAR 2011-2019, math operators)
• OpenCV (binarization, deskewing, connected components)
• SymPy (symbolic evaluation and solving)
• Jupyter notebook (evaluation with symbol-accuracy procedure)
• NumPy, matplotlib (confusion analysis and annotated outputs)

• CNN symbol classifier trained on EMNIST plus CROHME math symbols (around 100 classes)
• Connected-component segmentation with deskewing and size normalization
• Two-dimensional structural parser recovering superscripts, fractions and roots
• SymPy backend for evaluation, simplification and symbolic solving
• Annotated equation images showing segments, labels and confidence
• Equation-sheet mode: batch-solve a page of photographed equations
• Evaluation notebook with held-out symbol-accuracy procedure and solve-rate log
• Demo app for uploading equation photos and stepping through the parse

• Complete source code (preprocessing, CNN training, segmentation, parser, solver, demo app)
• Jupyter training and evaluation notebook (symbol accuracy, confusion analysis, solve-rate log)
• Project report PDF (background, EMNIST and CROHME, methodology, evaluation, error analysis)
• PPT presentation for final review
• Viva Q&A preparation document (CNNs, segmentation, structural parsing, SymPy, dataset coverage)
• Setup guide (environment, dataset download, training, running on your own equation photos)

• Recognition covers the symbol classes present in EMNIST and the CROHME symbol inventory: handwriting styles far outside that training mix classify worse, and the report documents this coverage gap openly.
• Heavily cursive writing, overlapping symbols and poor lighting break the segmentation stage before recognition even starts; the pipeline assumes clearly separated, legible handwriting.
• This is an educational prototype combining recognition with symbolic computation, not a certified math-assessment tool — results are advisory and should be verified independently.