Yoga Pose Classification CNN with Yoga-82 Dataset

Yoga poses are hard to recognize automatically because they are fine-grained: two poses can look nearly identical except for a hand or head position, bodies are frequently occluded, and the same pose looks different from every camera angle. Manual keypoint annotation breaks down on these complex postures, which is why the Yoga-82 authors reframed pose recognition as classification and built a dedicated 82-class dataset with a three-level label hierarchy. This project takes that dataset and trains a real transfer-learning CNN on it, so the student learns the full modern vision pipeline: working with a published academic dataset, fine-tuning a pre-trained backbone, evaluating with top-1/top-5 accuracy and per-class metrics, and deploying the trained model behind a web demo. The result is a defensible deep-learning project with a citable dataset, a documented training run, and a live classifier the examiner can try during the demo.

1. The Yoga-82 dataset is downloaded from its official release: 21,009 training and 7,469 test images across 82 pose classes, each image carrying a 3-level hierarchical label.
2. Images are resized to 224x224, normalized with ImageNet statistics, and augmented during training with random crops, horizontal flips, small rotations and color jitter.
3. A pre-trained ResNet-50 (or DenseNet-121) backbone is first trained with frozen weights on the new 82-way head, then unfrozen for full fine-tuning with cross-entropy loss and Adam.
4. The trained model is evaluated once on the held-out test set: top-1/top-5 accuracy, per-class precision/recall, and a confusion matrix are generated for the report.
5. The best checkpoint is exported and loaded by a Flask REST API that accepts an image and returns the ranked 82-class softmax output.
6. The web demo sends an uploaded photo or webcam frame to the API and renders the predicted pose, its Sanskrit name, group, description and top-3 confidence bars.

Tech stack:

• Python 3, PyTorch (torchvision)
• Yoga-82 dataset (official release)
• Flask REST API (model serving)
• OpenCV, Pillow (preprocessing)
• scikit-learn (metrics)
• Matplotlib, Seaborn (plots)
• Jupyter Notebook (training & evaluation)
• HTML, CSS, JavaScript (web demo)

Dataset & model details

• Dataset: Yoga-82 — over 28,400 yoga pose images collected from the web via Bing, manually cleaned and annotated; official split 21,009 train / 7,469 test; 82 classes with 64 (min) to 1,133 (max) images per class (~347 average); three-level label hierarchy 6 -> 20 -> 82 (body position -> variation -> pose). Released with the paper "Yoga-82: A New Dataset for Fine-grained Classification of Human Poses" (Verma, Kumawat, Nakashima, Raman; CVPR Workshops 2020; arXiv:2004.10362).
• Task: 82-class fine-grained image classification; input = 224x224x3 RGB image, output = probability distribution over the 82 pose classes (plus an auxiliary 6-way group prediction).
• Model: Transfer-learning CNN — ResNet-50 or DenseNet-121 backbone (ImageNet pre-trained), global average pooling -> dropout -> 82-way softmax head, with an auxiliary 6-way hierarchical head; trained with cross-entropy loss, Adam optimizer, frozen-backbone warmup followed by fine-tuning.
• Metrics: Top-1 and top-5 accuracy on the 7,469-image test set, per-class precision/recall/F1, confusion matrix. Design target: approximately 78% top-1 and 92% top-5 accuracy for the built-to-order training run. No accuracy is claimed as measured until the training run is executed for the order; the report documents the actual achieved figures.

• [82-class CNN classifier] A ResNet-50 (or DenseNet-121) backbone pre-trained on ImageNet, fine-tuned on Yoga-82's 82 pose classes with a 224x224 input and an 82-way softmax output.
• [Hierarchical classification head] An auxiliary head predicts the 6 top-level body-position groups (standing, sitting, balancing, inverted, reclining, wheel), guiding the fine-grained classifier on easily confused pose pairs — the approach introduced in the Yoga-82 paper.
• [Web inference demo] Upload a photo or capture a frame from the webcam; the app shows the predicted pose with its Sanskrit name, group, a short description, and top-3 predictions with animated confidence bars.
• [82-class pose browser] A searchable label browser of all 82 classes organized by the 6 official top-level groups, with English and Sanskrit names for every pose.
• [Full training notebook] Dataset loading and the 21,009/7,469 train/test split, augmentation (crop, flip, rotation, color jitter), frozen-backbone warmup, fine-tuning loop and checkpointing in one reproducible Jupyter notebook.
• [Evaluation suite] Top-1 and top-5 accuracy, per-class precision and recall, and a confusion matrix highlighting which pose pairs the model confuses most — ready for the report and the viva.
• [REST inference API] A Flask endpoint that loads the trained weights and returns ranked predictions as JSON, so the web demo and any client app share one serving layer.
• [Pose-practice feedback mode] Optional customization: side-by-side comparison of the user's photo against a reference image of the predicted pose, with notes on common alignment mistakes. Not part of the base build; developed on request.

• Complete training & evaluation Jupyter notebook (data pipeline, augmentation, training loop, metrics)
• Trained CNN weights (.pth) with the preprocessing and label files
• Flask REST inference API with request/response documentation
• Web demo: upload/webcam classification, top-3 confidences, 82-class browser, dataset & model panel
• Evaluation artifacts: accuracy numbers, per-class report, confusion matrix, training-curve plots
• Project report PDF (background, dataset analysis, architecture rationale, methodology, results)
• PPT presentation for final review
• Viva Q&A preparation document (CNNs, transfer learning, softmax, overfitting, hierarchical labels)