SSUET final-year project · AI + accessibility

Hands speak.
Narvi gives them a voice.

A real-time sign language translator that watches both hands, understands the gesture in milliseconds, and speaks it aloud in English or اردو — right in your browser.

Try it live How it works

26: gesture classes; 20 trained + 6 two-hand
86%: test accuracy; held-out evaluation
~1.5ms: classification; landmark fast-path
2: hands tracked; MediaPipe, live

lm[8] · index_tiplm[4] · thumb_tiplm[20] · pinky_tiplm[0] · wrist

Hello!السلام علیکمHow are you?آپ کیسے ہیں؟Thank youشکریہI love youمیں آپ سے محبت کرتا ہوںHelpمجھے مدد چاہیےNamasteآدابWell done!شاباشGood morningصبح بخیرWaterپانیCoffeeکافیStopرک جائیںWe are togetherہم ساتھ ہیںHello!السلام علیکمHow are you?آپ کیسے ہیں؟Thank youشکریہI love youمیں آپ سے محبت کرتا ہوںHelpمجھے مدد چاہیےNamasteآدابWell done!شاباشGood morningصبح بخیرWaterپانیCoffeeکافیStopرک جائیںWe are togetherہم ساتھ ہیں

Why Narvi

Conversation shouldn't need a translator in the room

Millions of deaf and hard-of-hearing people in Pakistan sign every day to families, shopkeepers and classmates who never learned to sign back. Narvi sits in the middle of that conversation — a camera, a model, and a voice — so signing is understood the moment it happens, in both of the languages people here actually speak.

Two-hand recognition

MediaPipe tracks up to two hands at once. Single-hand signs hit the trained model; two-hand gestures like namaste and clap resolve through a geometric rule engine — or natively once the 126-dim model is trained.

Millisecond classification

The browser sends 63–126 landmark floats instead of JPEG frames. No re-detection on the server, no image decode — around 1.5 ms per prediction.

English + اردو, spoken

Every gesture maps to both languages. English speaks through the browser; Urdu speaks through server-side TTS with native pronunciation.

Sentence building

Hold a gesture until the ring fills to confirm it. Confirmed phrases chain into full sentences you can replay in either language.

Offline fallback

If the backend is unreachable, a built-in rule engine keeps a core set of one- and two-hand gestures working entirely in the browser.

Mirror-proof by design

Every prediction is also evaluated horizontally mirrored and the more confident result wins — camera orientation can't break recognition.

From gesture to voice

Five steps, about a blink and a half

Camera

The webcam feed stays in the browser. Frames never leave your device.

Landmarks

MediaPipe Hands finds 21 joints per hand — up to 42 points, ~30× a second.

Normalize

Each hand is re-centred on the wrist and scaled by hand size: position and distance invariant.

Classify

A dense neural network scores all classes in ~1.5 ms. Low-confidence frames are discarded.

Speak

The confirmed gesture is voiced in English or Urdu and appended to the sentence.

“Every gesture already means something.
We just make it audible.”

— Team Narvi

No install · runs in your browser

Raise a hand. Hear it speak.

Open the live translator, allow the camera, and hold any gesture until the ring fills. That's the whole tutorial.

Launch the translator

Hands speak.Narvi gives them a voice.