Virtual Try-On using Pose Estimation

Online clothing shoppers face a basic problem: product photos show the garment, never the garment on them. Full 3D virtual try-on is a research-grade problem — cloth simulation, body-shape modeling, lighting relighting — far beyond a student build. But a convincing 2D approximation is very much in reach: find the person's pose in the photo, pin the garment image to the right body landmarks, warp it to follow the torso, and composite it with proper alpha blending. The result reads immediately as "me wearing that shirt," and the whole pipeline runs on a laptop with no training at all.

This project builds that approximation with MediaPipe Pose, which returns 33 body landmarks from a single photo. The buyer prepares garment images as PNGs with transparent backgrounds; at try-on time the garment is anchored to shoulder, elbow and hip landmarks, warped with a perspective transform to follow the pose, and blended onto the photo. Everything is qualitative — there is no accuracy number for how good a try-on looks, so the report evaluates with before/after figures and a landmark-alignment study. The limitations section is explicit: this is a flat 2D warp, so fit is approximate and true drape, folds and fabric physics are not simulated.

1. The user uploads a photo; MediaPipe Pose detects 33 body landmarks.
2. Shoulder width and torso length are estimated from the landmark coordinates.
3. The selected garment PNG is scaled to those body measurements.
4. Anchor points on the garment (shoulders, sleeve ends, hem) are matched to the corresponding landmarks.
5. A perspective transform warps the garment to follow the detected torso pose.
6. The warped garment is alpha-blended onto the photo with feathered edges, and the result is displayed and saved.

Tech stack:

• Python 3.10, MediaPipe (pose landmark detection)
• OpenCV (warping, alpha blending, image compositing)
• NumPy (landmark geometry and transform math)
• Perspective transform via cv2.getPerspectiveTransform
• PNG garments with transparent backgrounds (buyer-prepared library)
• Flask demo app (photo upload and garment picker)
• Matplotlib (landmark-alignment figures for the report)

• MediaPipe Pose landmark detection (33 body keypoints) from a single photo
• Garment library: PNG garments with transparent backgrounds, anchored to landmarks
• Perspective-transform warping that follows shoulder, torso and hip pose
• Alpha-blended compositing with soft edge feathering
• Garment scaling from shoulder-width and torso-length measurements
• Sleeve anchoring to elbow and wrist landmarks
• Before/after comparison view and try-on image export
• Garment-preparation guide for adding new items to the library

• Complete source code (pose detection, garment warping, compositing, demo app)
• Demo app with garment library and photo upload
• Sample garment PNG set with transparent backgrounds
• Project report PDF (background, pose estimation, warping method, qualitative evaluation, limitations)
• PPT presentation for final review
• Viva Q&A preparation document (MediaPipe, landmarks, perspective transforms, compositing, honest scope)
• Setup guide (environment, garment preparation, running the demo)

• This is a flat 2D warp, not cloth simulation: there is no true 3D drape, no folds, no fabric physics, and fit is approximate — the report states this openly rather than overselling the effect.
• Pose quality bounds everything: side views, crossed arms and occluded shoulders break landmark detection and with it the garment alignment.
• Garment PNGs must be prepared with clean transparent backgrounds and roughly front-facing cuts; the guide covers this, but the library quality is the buyer's own work.
• Lighting and shadow on the garment will not match the photo, so the composite reads as a visualization aid, not a photograph of the real garment worn.