Human Activity Recognition using 3D CNN

Security teams, sports analysts and smart-camera builders all want the same thing from a video stream: a plain-language answer to "what is happening?" — walking, running, sitting. Manually reviewing video footage does not scale: nobody can watch every hour of footage, and a 2D CNN that sees one frame at a time is blind to the motion that defines an activity. A single still of a person mid-stride cannot distinguish walking from running; the answer lives in how the frames change over time. A 3D CNN addresses exactly this by convolving across height, width and time together, so each filter learns motion patterns — a stride, a swinging arm — directly from short frame sequences. For students, the project fills the gap between image classification and true video understanding: it demonstrates clip sampling, spatiotemporal augmentation, closed-set action classification and an honest per-class confusion analysis, all on the standard academic benchmark for the task.

Frequently asked questions

1. What is a 3D CNN and why is it used here? A 3D CNN applies convolution across height, width and time, so each filter learns motion patterns (a swinging arm, a stride) directly from stacked frames. For activity recognition this beats single-frame 2D CNNs, which cannot see motion at all.
2. What dataset does the model train on? UCF101 — 13,320 real-world YouTube video clips across 101 action categories, introduced by Soomro et al. (2012). It is the standard academic benchmark for action recognition, and the report documents the train/validation split used in the build.
3. How is the model evaluated? The training notebook computes clip-level accuracy on the validation split every epoch, logs accuracy and loss curves, and builds a per-class confusion matrix — the report documents the procedure and the metrics the build's own training run computes.
4. How long should an uploaded clip be? A few seconds is enough: the demo samples a fixed-length frame sequence from whatever is uploaded, so short clips work best.
5. Is this project suitable for a final-year project? Yes. It suits B.E./B.Tech students in Computer Science, AI/ML and Data Science, demonstrating 3D convolutions, video preprocessing, temporal augmentation, action-recognition evaluation and a clip-upload demo.
6. What will I receive? Complete source code, trained 3D-CNN weights (.pth), dataset download and split-configuration scripts, project report PDF, PPT presentation, viva Q&A document and a setup guide.

Dataset & model:
Dataset name: UCF101.
Source: publicly released action-recognition benchmark (Soomro et al., 2012).
Task: closed-set video action classification.
Classes: 101 action categories across 13,320 real-world YouTube clips.
Model: C3D-style 3D CNN (Tran et al., arXiv:1412.0767), trained in PyTorch.
Input: short video clip, sampled into a fixed-length frame sequence and stacked into a spatiotemporal tensor.
Prediction: 101-way softmax over action classes.
Output: top-3 predicted activities with confidence bars.
Evaluation metrics: clip-level accuracy, loss curves, per-class confusion matrix — computed by the training notebook on the validation split during the build.
Design target: ~85%+ clip accuracy on the validation split. This is a design target, not a measured claim; the notebook computes the actual accuracy during the build.

Working:

1. Training phase: UCF101 clips are decoded and sampled into fixed-length frame sequences, split by video group into train and validation sets; training clips are augmented with random crops, horizontal flips and temporal jitter, while validation clips are center-cropped.
2. A C3D-style 3D CNN trains on clip batches; every epoch logs clip accuracy and loss curves on the validation split and saves the best checkpoint.
3. Evaluation phase: the training notebook computes clip-level accuracy and builds a per-class confusion matrix on the validation split, documenting which activities are confused with which.
4. Inference phase: the Flask app loads the best .pth weights at startup; each uploaded clip is sampled to a fixed-length sequence, normalized and stacked into a spatiotemporal tensor for one forward pass.
5. Softmax scores are ranked and the top-3 activities are displayed with confidence bars.
6. The classifier is closed-set over the UCF101 classes: out-of-distribution actions get the nearest-looking label, which the Limitations section documents openly.

Specifications:
Model | C3D-style 3D CNN trained on UCF101
Dataset | UCF101 — 13,320 YouTube clips, 101 action classes (Soomro et al. 2012)
Metric | Clip accuracy on the validation split, computed by the training notebook during the build
Design target | ~85%+ clip accuracy (design target, not a measured claim)
Input | Short video clips, auto-sampled to fixed-length frame sequences
Demo app | Flask web app with clip upload and top-3 prediction display
Weights | .pth file shipped with the build

[3D-CNN Action Classifier] (implemented) — A C3D-style 3D convolutional network classifies UCF101 action classes from short clips, served through a Flask demo app.
[Top-3 Clip Predictions] (implemented) — Upload a clip and get the three most likely activities with confidence bars.
[Clip Sampling & Augmentation Pipeline] (implemented) — Fixed-length frame sampling with random crops, horizontal flips and temporal jitter during training.
[Training & Evaluation Notebook] (implemented) — Logs clip accuracy and loss curves every epoch and computes the per-class confusion matrix on the validation split during the build.
[Per-Class Evaluation] (implemented) — The confusion matrix documents exactly which activities are confused with which — honest, discussable error analysis.
[Exportable .pth Weights] (implemented) — The best checkpoint ships as a .pth file for deployment-style demos.
[Configurable Top-k Display] (configurable) — The number of displayed activities and the confidence display can be adjusted in the app.
[Video-Group Split Scripts] (implemented) — Dataset download and split-configuration scripts enforce the by-video-group train/validation split the benchmark expects.

Complete source code (data pipeline, training, evaluation, Flask app)
Trained 3D-CNN weights (.pth)
Dataset download and split-configuration scripts
Project report PDF (background, dataset analysis, methodology, evaluation, error analysis)
PPT presentation for the final review
Viva Q&A preparation document (3D convolutions, action recognition, augmentation, evaluation metrics)
Setup guide (environment, dependencies, GPU guidance for training)