Yoga Pose Estimation and Correction using MediaPipe

Most yoga learners practise alone — in a hostel room or at home — with no one watching their alignment. A knee collapsing inward in Warrior II or sagging hips in Downward Dog quietly turns a strengthening pose into a strained joint, and by the time a teacher sees it, the habit is set. Existing fitness apps count repetitions or track calories, but almost none check whether the pose itself is geometrically right. This project closes that gap with on-device computer vision: MediaPipe Pose detects 33 body landmarks per frame, the application derives joint angles from landmark triplets, and an alignment engine compares each angle against a reference table for the selected asana. The result is immediate, specific feedback on form — the part of yoga instruction that is hardest to get without a teacher in the room.

1. A video frame from the webcam — or a sample/uploaded photo — is passed to the MediaPipe PoseLandmarker (tasks-vision), which returns 33 normalized landmarks with per-point visibility.
2. The app converts the normalized landmarks to pixel coordinates and computes joint angles (elbow, knee, hip, shoulder) from landmark triplets using the cosine rule.
3. The selected asana's reference table supplies a target angle and tolerance for each checked joint (design targets from standard alignment teaching, e.g. 90° ± 12° for the Warrior II front knee).
4. Each measured angle is compared against its target; deviations inside ±12° pass, deviations outside become correction cues with the measured value, the target and a concrete fix.
5. The skeleton overlay is drawn on the frame — green segments for joints within tolerance, amber for joints needing correction — with degree pills on the primary joints.
6. The posture score (100 minus weighted deviations) and the hold timer update live, and the session summary can be copied for a practice log.

Tech stack:

• MediaPipe Pose · BlazePose GHUM 3D (pre-trained)
• @mediapipe/tasks-vision (PoseLandmarker, CDN)
• JavaScript (ES2020) + HTML5 Canvas 2D overlay
• WebRTC getUserMedia (webcam capture)
• Python 3 (report/abstract tooling)
• Git

Dataset & model details

• Dataset: None in this project — MediaPipe Pose is used pre-trained exactly as shipped by Google. There is no project-specific training set, no labeling effort and no training run to report.
• Task: Human pose landmark regression; input = an RGB video frame, output = 33 body landmarks (x, y, visibility) per detected person. The project's own computation is joint-angle derivation and rule-based alignment checking on top of those landmarks.
• Model: BlazePose GHUM 3D (MediaPipe Pose), pre-trained by Google. Used unmodified via the tasks-vision PoseLandmarker API with GPU delegate where available.
• Metrics: No accuracy is claimed for the model. The project reports approximate joint angles with a ±12° acceptance tolerance (design target) and a computed 0–100 posture heuristic — both are engineering choices, not measured results.

• [Real-time 33-landmark pose detection] MediaPipe Pose (BlazePose GHUM 3D) runs on each video frame and returns 33 body landmarks — shoulders, elbows, wrists, hips, knees, ankles and facial points — entirely on-device in the browser. The model is used pre-trained as shipped by Google; this project trains nothing and claims no accuracy figure.
• [Joint-angle computation] Elbow, knee, hip and shoulder angles are derived from landmark triplets with the cosine rule and reported in degrees. Angles are 2D projections from a single camera and are presented as approximate — the app never pretends to measure clinical-grade biomechanics.
• [Asana reference-alignment engine] Each supported asana ships with a reference table of target angles taken from standard yoga alignment teaching — for example, a 90° front knee in Virabhadrasana II or straight 180° arms in Adho Mukha Svanasana — checked inside a ±12° design tolerance.
• [Plain-language correction feedback] Any angle outside tolerance becomes a specific cue naming the joint, the measured value and the fix, such as "Front knee: 117° (target 90°) — sink deeper until the thigh is parallel to the floor." Joints needing correction are highlighted amber on the skeleton overlay; correct joints stay green.
• [Posture score] A 0–100 score computed as 100 minus weighted angle deviations, so a session has one number to improve across attempts. It is a computed heuristic, not a measured performance claim.
• [Hold timer with session best] A start/pause/reset timer tracks how long the pose is held, records the session best, and pairs naturally with the posture score for practice sessions.
• [Sample-photo analysis mode] Three built-in sample photos (Vrikshasana, Virabhadrasana II, Adho Mukha Svanasana — CC-licensed, credited in-app) demonstrate the full analysis pipeline offline, with reference landmarks calibrated per photo so the UI works without a camera.
• [Reference alignment library] A browsable table of every target angle, tolerance and alignment cue per asana, plus "How it works" and "Model & honesty" panels that document the pipeline and its limits in plain terms.

• Pose-analysis web app (single-file HTML/CSS/JS): live webcam mode, sample-photo analysis mode, skeleton overlay with angle pills, corrections panel, joint-angle table, posture score, hold timer
• Asana reference-alignment library for 3 asanas (target angles, tolerances, alignment cues) with "How it works" and "Model & honesty" documentation panels
• MediaPipe tasks-vision integration code (PoseLandmarker setup, GPU delegate, video-frame loop)
• Project report PDF (background, BlazePose overview, methodology, alignment-table rationale, limitations)
• PPT presentation for final review
• Viva Q&A preparation document (pose estimation vs classification, landmark topology, 2D angle limits, why pre-trained)