Shoplifting Detection using Pose Estimation

Retail shrinkage costs stores real money, and reviewing hours of CCTV to find one concealment gesture is exactly the kind of task humans do badly: attention fades, footage piles up, and incidents are found hours too late. Pose estimation offers a different angle from raw video classification — instead of asking "does this frame look like theft?", it tracks what the body is doing: where the hands go, how long they stay, whether a product-handling motion is followed by a pocket-zone motion. This project builds that pipeline: MediaPipe pose extracts real-time body landmarks, a tracking layer follows hand and wrist positions across frames, and a gesture-heuristics engine scores concealment-like patterns sustained over a time window. The output is deliberately an advisory alert log for human review, never an accusation — the honest framing for a student system with a high false-positive rate, and the report documents the privacy and ethics boundary explicitly.

1. Store footage frames are sampled and MediaPipe pose extracts body landmarks for each visible person.
2. Hand and wrist landmarks are tracked across frames; dwell time in torso and pocket zones is computed per person.
3. The gesture-heuristics engine scores concealment-like behaviors: hand-to-torso contact with rapid retraction, repeated reaching motions, product-handling followed by pocket-zone motion.
4. Gesture patterns sustained over a time window become advisory alerts with a timestamped log entry and a frame crop.
5. The demo app presents alerts as a review queue — a human watches each flagged segment and decides, because the system never accuses.
6. The notebook runs the staged evaluation: labeled actors perform normal shopping and concealment scenarios, and the alert rule's precision and recall are computed.

Tech stack:

• Python 3.10, OpenCV (video capture, landmark overlays, frame crops)
• MediaPipe pose (real-time body landmarks, Lugaresi et al.)
• Gesture heuristics engine (hand-to-torso rules, dwell times, time windows)
• NumPy, pandas (gesture features and alert logs)
• Jupyter notebook (staged-footage evaluation procedure)
• Matplotlib (gesture timelines and evaluation plots)
• Flask demo app with footage upload and alert-review interface

• MediaPipe pose extracting real-time body landmarks per visible person
• Hand and wrist tracking across frames with dwell-time measurement
• Gesture heuristics: hand-to-torso contact, rapid retraction, repeated reaching, product-then-pocket patterns
• Time-window alert rule: alerts only on sustained gesture patterns
• Advisory alert log with timestamps and frame crops for human review
• Review-oriented demo app: browse alerts, jump to the flagged footage segment
• Buyer-run evaluation procedure on staged footage with labeled actors
• Notebook computing the alert rule's precision and recall on the staged set

• Complete source code (pose pipeline, gesture engine, alert logic, review app)
• Jupyter notebook with the buyer-run staged-footage evaluation procedure
• Project report PDF (background, pose methodology, gesture heuristics, evaluation, ethics and limits)
• PPT presentation for final review
• Viva Q&A preparation document (MediaPipe, landmark tracking, gesture design, precision/recall)
• Setup guide (environment, camera angles, staging the evaluation footage, running the pipeline)

• The alerts are advisory only, NOT evidence: the system flags gestures that merely resemble concealment, with a high false-positive risk, and must never be used to accuse, detain or report any real person.
• Testing must use staged footage with actors or consenting-subject footage only — running it on real customers without consent and notice is a privacy violation, and the report states this ethics boundary explicitly.
• Occlusion, crowded aisles, loose clothing and camera angles outside the staged evaluation degrade landmark quality; the pipeline flags what the camera sees, never intent.