A camera, twenty-one points per hand, and 1.5 milliseconds of math
Narvi never sends your video anywhere. The browser reduces each frame to hand-landmark coordinates; a compact neural network turns those coordinates into words; your speakers do the rest.
Camera
The webcam feed stays in the browser. Frames never leave your device.
Landmarks
MediaPipe Hands finds 21 joints per hand — up to 42 points, ~30× a second.
Normalize
Each hand is re-centred on the wrist and scaled by hand size: position and distance invariant.
Classify
A dense neural network scores all classes in ~1.5 ms. Low-confidence frames are discarded.
Speak
The confirmed gesture is voiced in English or Urdu and appended to the sentence.
Small network, serious numbers
Perception
- MediaPipe Hands
- 21-point landmarks
- 2-hand tracking
Model
- TensorFlow / Keras
- Dense NN · 126-dim input
- TFLite export
Backend
- Flask REST API
- /classify-landmarks fast path
- gTTS Urdu voice
Frontend
- Next.js 14
- Framer Motion
- Tailwind CSS
What changed when we tore out the spine
Two hands, end to end
Collection, preprocessing, training and the live app all understand a second hand. Old single-hand data is auto-padded, so nothing collected in v1 was wasted.
40–100× faster classification
v1 shipped ~50 KB JPEG frames and re-ran hand detection on the server. v2 sends the landmark floats the browser already has — the server just normalizes and predicts.
Rules where models are blind
A geometric rule engine resolves two-hand gestures with strict spatial conditions, and acts as a high-precision first pass even after the two-hand model is trained.
Safe to retrain
The server auto-detects the model's input size and refuses mismatched scaler/model pairs — a half-finished retrain can never silently produce wrong predictions.
Numbers are nice. The demo is better.
See the whole pipeline run against your own hands, live.