flutter_hand_gesture

Real-time hand gesture recognition for Flutter — fully on-device, no internet required, no API key needed.

Features

✋ 10 built-in gestures — fist, open palm, thumbs up/down, peace, pointing, rock, call me, OK, vulcan
🎯 21-point hand landmarks — full skeleton tracking
🧠 Custom gesture training — teach your own gestures with KNN classifier
📡 Stream-based API — reactive, works with BLoC / Provider / Riverpod
🔇 Fully offline — TFLite models run on-device
🎨 Plug-and-play widget — GestureCameraView with landmark overlay
🤲 Multi-hand support — detect up to 2 hands simultaneously
⚡ Background isolate — zero UI thread blocking

Quick Start

// 1. Plug-and-play widget
GestureCameraView(
  onGestureDetected: (gesture) {
    print('${gesture.emoji} ${gesture.displayName}');
    // 👍 Thumbs Up
  },
)

// 2. Stream-based (headless)
final detector = HandGestureDetector(
  config: GestureConfig(minConfidence: 0.8),
);
await detector.initialize();
await detector.startDetection();

detector.gestureStream.listen((result) {
  final gesture = result.primaryGesture;
  if (gesture != null) {
    print(gesture.type); // GestureType.thumbsUp
  }
});

// 3. Custom gesture training
final trainer = CustomGestureTrainer();
await trainer.load();

// Training (collect 5+ samples per gesture)
await trainer.addSample('namaste', landmarks);

// Prediction
final label = trainer.predict(landmarks); // 'namaste'

Supported Gestures

Gesture	Emoji	GestureType
Fist	✊	`GestureType.fist`
Open Palm	🖐	`GestureType.openPalm`
Thumbs Up	👍	`GestureType.thumbsUp`
Thumbs Down	👎	`GestureType.thumbsDown`
Peace	✌️	`GestureType.peace`
Pointing	👆	`GestureType.pointing`
Rock Sign	🤘	`GestureType.rockSign`
Call Me	🤙	`GestureType.callMe`
OK Sign	👌	`GestureType.okSign`
Vulcan	🖖	`GestureType.vulcanSalute`
Custom	🤚	`GestureType.custom`

Installation

dependencies:
  flutter_hand_gesture: ^0.1.0

Android

Add to android/app/src/main/AndroidManifest.xml:

<uses-permission android:name="android.permission.CAMERA" />

Set minSdkVersion to 24 in android/app/build.gradle(.kts) (required by the on-device ML backend):

minSdk = 24

iOS

Add to ios/Runner/Info.plist:

<key>NSCameraUsageDescription</key>
<string>Camera is used for hand gesture recognition</string>

Configuration

GestureConfig(
  minConfidence: 0.75,        // 0.0 - 1.0
  maxHands: 1,                // 1 or 2
  showLandmarks: true,        // draw skeleton overlay
  targetGestures: [           // filter to specific gestures only
    GestureType.thumbsUp,
    GestureType.peace,
  ],
  debounceDuration: Duration(milliseconds: 500),
  mirrorCamera: true,         // for front camera
  useBackgroundIsolate: true, // keep UI smooth
)

API Reference

`HandGestureDetector`

Method	Description
`initialize()`	Set up camera and ML model
`startDetection()`	Begin processing frames
`stopDetection()`	Pause processing
`gestureStream`	`Stream<GestureResult>`
`landmarkStream`	`Stream<List<HandLandmarks>>`
`dispose()`	Release all resources

`GestureCameraView`

Prop	Type	Description
`config`	`GestureConfig`	Detection settings
`onGestureDetected`	`(HandGesture) → void`	Called on each gesture
`onResult`	`(GestureResult) → void`	Full frame result
`showGestureLabel`	`bool`	Show label overlay
`showLandmarks`	`bool`	Show skeleton overlay
`overlayBuilder`	`Widget Function(context, result)`	Custom overlay

`CustomGestureTrainer`

Method	Description
`addSample(label, landmarks)`	Add training sample
`predict(landmarks)`	Predict label
`predictWithConfidence(landmarks)`	Predict with score
`save()`	Persist to disk
`load()`	Load from disk
`clearLabel(label)`	Remove a gesture

Roadmap

x TFLite MediaPipe model integration (v0.2.0) — via hand_detection
Tamil sign language preset (v0.3.0)
Gesture sequence recognition (v0.3.0)
Web platform support (v0.4.0)
macOS / Windows support (v0.5.0)