More project photos (3)
The problem
Cameras are the obvious way to recognize gestures, but they are power-hungry, privacy-sensitive and useless in the dark. An inertial measurement unit on the wrist captures the same intent through motion: every gesture has a distinctive acceleration and rotation signature. The catch is running a neural network on a microcontroller with kilobytes of RAM — which is exactly what TinyML is for. This project builds the complete pipeline: a wrist strap carrying an IMU (Arduino Nano 33 BLE Sense or ESP32 + MPU6050), a serial capture tool for recording labeled 2-second gesture windows, a compact 1-D CNN trained on the collected data, int8 quantization that shrinks the model to tens of kilobytes, and on-device inference with a live test console. The console streams the IMU waveforms, shows per-class confidence bars for the four gestures, and presents the confusion matrix computed on the held-out test set — so every metric in the report comes from the student's own data and training run, never from a pre-claimed figure.
Wears the wrist IMU and performs fist, open palm, wave, or tilt.
Follow the working sequence
How it works
- The wrist-worn IMU samples accelerometer and gyroscope at 104 Hz while the user performs a gesture.
- The capture tool slices the stream into 2-second windows (208 samples × 6 axes) with 50% overlap and normalizes each axis.
- Labeled windows are collected per gesture — the guide targets at least 100 windows per class across 2–3 people.
- A compact 1-D CNN (two convolution blocks + dense head) is trained in TensorFlow/Keras on the collected set with a held-out test split.
- The trained model is fully quantized to int8 and converted to a C array for the microcontroller build.
- On-device, each new window runs through the quantized model in about 12 ms; the class with the highest score wins.
- The live console mirrors the same pipeline on the PC: streaming waveforms, confidence bars and the test-set confusion matrix.
Tech stack:
- IMU: Arduino Nano 33 BLE Sense (on-board LSM9DS1) or ESP32 + MPU6050
- TensorFlow/Keras (1-D CNN training), full-int8 quantization
- Serial data-collection tool (Python) for labeled window capture
- On-device inference firmware (Arduino core)
- Single-file HTML/CSS/JS live console (stream, confidences, confusion matrix)
- Custom dataset collected with the included tool (4 classes)
Dataset & Model Details
| Parameter | Value |
|---|---|
| Dataset | Custom IMU gesture set collected with the included capture tool — 4 classes (fist, open palm, wave, tilt) |
| Collection | 2-second windows at 104 Hz, 50% overlap, per-axis normalization; target ≥100 windows/class across 2–3 people |
| Split | Train/test split (e.g. 464 train / 120 test) defined in the notebook |
| Model | 1-D CNN, two convolution blocks + dense head |
| Input | 208 × 6 (accelerometer + gyroscope axes) |
| Output | 4-class softmax (fist, open palm, wave, tilt) |
| Quantization | int8 full-integer; ~41 KB flash, ~18 KB RAM |
| Inference | Design target ~12 ms per window on nRF52840-class MCU |
| Evaluation | Accuracy, per-class precision/recall, confusion matrix — computed by the notebook on the held-out test set |
Project features
- [On-device inference] The quantized int8 model runs entirely on the microcontroller — no phone, no cloud, no camera; a gesture is classified in roughly 12 ms per window.
- [Four-gesture classifier] Fist, open palm, wave and tilt, chosen so the set spans static poses and dynamic motions for interesting viva discussion.
- [Data-collection tool] Serial capture utility records labeled 2-second windows at 104 Hz with 50% overlap and per-axis normalization — the same tool the buyer uses to extend the dataset.
- [int8 quantization] Full-integer quantization shrinks the model to about 41 KB flash / 18 KB RAM, with the accuracy-vs-size tradeoff documented in the report.
- [Live test console] Streams accelerometer waveforms, renders per-class confidence bars in real time and shows reference captures for each gesture class.
- [Confusion matrix view] Held-out test results presented per class, so the known wave/tilt confusion is visible and discussable rather than hidden.
- [Custom gesture extension] The notebook documents how to record a new class and retrain — the supported path for college-specific customization.
- [Low-power design] IMU duty-cycling and sleep between windows are documented for battery-powered wearable use.
What is included
- Serial data-collection tool (labeled window capture at 104 Hz)
- Training notebook (1-D CNN, quantization, evaluation, confusion matrix)
- Quantized int8 model as a C array for firmware
- On-device inference firmware (wrist IMU build)
- Live test console (IMU stream, confidence bars, confusion matrix view)
- Data-collection guide (window counts, multi-person protocol)
- Project report PDF (background, TinyML pipeline, quantization tradeoff, evaluation)
- PPT presentation for final review
- Viva Q&A preparation document (IMU axes, CNNs for time series, quantization, overfitting, evaluation)
- Setup guide (strap assembly, capture tool, training, flashing, console)
Limitations & prerequisites
- Accuracy is measured on the student's own collected data — the report documents the procedure and the confusion matrix, not a pre-claimed percentage; results vary with how carefully data is collected.
- Wave and tilt confuse occasionally because both are dominated by x-axis motion; the report shows this per-class result openly as a known limitation.
- The model recognizes only the four trained gestures; anything else is forced into one of the four classes — there is no "unknown" rejection in the base build.
- Performance depends on consistent wearing position; a loose strap or a different wrist changes the signal and the guide says so.
- Training needs a few hundred clean windows — sloppy, rushed collection gives a sloppy model, and the guide's multi-person protocol exists for this reason.
- This is a prototype wearable, not a certified input device; latency and robustness suit demos and assistive-tech experiments, not safety-critical control.
Frequently Asked Questions
Which dataset is used?
A custom one you collect with the included capture tool: 2-second IMU windows at 104 Hz for fist, open palm, wave and tilt, with a target of at least 100 windows per class across 2–3 people. There is no public download — the dataset is part of the build.
Which model is used?
A compact 1-D CNN (two convolution blocks plus a dense head) trained in TensorFlow/Keras, then fully quantized to int8 for the microcontroller.
Is the accuracy guaranteed?
No. Every metric — accuracy, per-class precision/recall, confusion matrix — is computed by the training notebook on your held-out test set during your build. The report documents your numbers.
Can I add my own gestures?
Yes — that is the documented extension path: record windows for the new class with the capture tool, retrain the notebook, re-quantize and reflash. The guide walks through it.
Why not just use a camera?
Cameras need more power and compute, raise privacy concerns and fail in the dark. An IMU is tiny, cheap and always on — the tradeoff is a smaller gesture vocabulary, which the report discusses.
Is this project suitable for a final-year project?
Yes — for AI/ML, Computer Science, Electronics and IoT programs. It demonstrates time-series deep learning, quantization, embedded deployment and honest evaluation, all strong viva material. Suitable for B.E./B.Tech final-year projects in AI/ML, Computer Science, Electronics and IoT.
Components & software requirements
- IMU: Arduino Nano 33 BLE Sense (on-board LSM9DS1) or ESP32 + MPU6050
- TensorFlow/Keras (1-D CNN training), full-int8 quantization
- Serial data-collection tool (Python) for labeled window capture
- On-device inference firmware (Arduino core)
- Single-file HTML/CSS/JS live console (stream, confidences, confusion matrix)
- Custom dataset collected with the included tool (4 classes)
Dataset & Model Details
| Parameter | Value |
|---|---|
| Dataset | Custom IMU gesture set collected with the included capture tool — 4 classes (fist, open palm, wave, tilt) |
| Collection | 2-second windows at 104 Hz, 50% overlap, per-axis normalization; target ≥100 windows/class across 2–3 people |
| Split | Train/test split (e.g. 464 train / 120 test) defined in the notebook |
| Model | 1-D CNN, two convolution blocks + dense head |
| Input | 208 × 6 (accelerometer + gyroscope axes) |
| Output | 4-class softmax (fist, open palm, wave, tilt) |
| Quantization | int8 full-integer; ~41 KB flash, ~18 KB RAM |
| Inference | Design target ~12 ms per window on nRF52840-class MCU |
| Evaluation | Accuracy, per-class precision/recall, confusion matrix — computed by the notebook on the held-out test set |
Delivery information
Built-to-order project. Delivery timeline is shared after order confirmation based on current queue.
Support terms
Complete documentation, setup guide, and viva preparation included. Support for setup and explanation provided.
Download free synopsis (PDF)
Enter your name and WhatsApp number — the synopsis PDF downloads immediately. We may send occasional project updates on WhatsApp; no spam.