पैकेज, रिपॉज़िटरी और ऐप श्रेणियाँ खोजें

GitHub रिपॉज़िटरीDeep-SkyLabs/gaze-control

दृष्टि नियंत्रण

Gaze Control एक एंड्रॉइड एक्सेसिबिलिटी उपकरण है जो उपयोगकर्ताओं को केवल आँखों, सिर के हिलने-डुलने या हथेली के चलने के बाद सोशल मीडिया फीड (Instagram, TikTok, YouTube Shorts, X/Twitter, Reddit) को स्क्रॉल करने की अनुमति देता है

दृष्टि नियंत्रण प्रोजेक्ट कवर
स्टार
3
फ़ोर्क
2
अंतिम पुश (UTC)
2 जून 2026
प्रोजेक्ट स्थिति
सक्रिय

मूल README

यह अंग्रेज़ी मूल स्नैपशॉट है। नवीनतम सामग्री GitHub पर देखें।

Gaze Control Banner

👁️ Gaze Control

Hands-free scrolling for Android — powered by real-time on-device eye tracking

Flutter Kotlin License PRs Welcome Platform MediaPipe

Read the Full Blog Post → · Report a Bug · Request a Feature


Gaze Control Demo

Table of Contents


Overview

Gaze Control is an Android accessibility utility that lets users scroll social media feeds — Instagram, TikTok, YouTube Shorts, X/Twitter, Reddit — using only their eyes, head movements, or palm gestures.

It runs as an Android Foreground Service, processes all camera frames locally on-device via a Google MediaPipe + L2CS-Net ONNX pipeline, and dispatches physical scroll gestures through the Android Accessibility Service API. No data ever leaves the device.

Built for people who want hands-free mobile interaction: those recovering from surgery, living with motor disabilities, multitasking, or simply curious about what gaze-driven interfaces feel like.


Features

Capability Description
👁️ Eye Gaze Tracking Look up/down past a configurable dwell threshold to trigger scrolls
🤙 Head Nod State Machine Subtle vertical/horizontal nods fire instant scroll actions via gyroscope fusion
🖐️ Palm Gesture Control Open Palm = Play/Pause · Closed Fist = Stop · Thumbs Up = Select
📐 9-Stage Calibration Personalized neutral anchors, gaze extents, deadzones, and directional biases
🔒 Full On-Device Privacy Camera frames processed in native memory, never stored or transmitted
🔋 Thermal-Aware Performance Dynamic frame throttling based on device temperature and tracking confidence
⚡ NNAPI Acceleration L2CS-Net inference runs with hardware NPU acceleration + math fallback
🛡️ Blink-Safe Tracking Eye Aspect Ratio + blendshape fusion prevents involuntary blinks from triggering actions

Architecture

The system is split into two primary layers communicating via Flutter platform method channels.

┌─────────────────────────── Flutter (Dart) ─────────────────────────────┐
│                                                                          │
│   GoRouter ──► Dashboard ──► Calibration ──► Settings ──► Onboarding   │
│                    │               │              │                      │
│              Riverpod NotifierProviders ◄── Platform Channel Adapters   │
│                                                                          │
└──────────────────────────────────┬───────────────────────────────────── ┘
                                   │ MethodChannel / EventChannel
┌──────────────────────────── Kotlin (Android) ──────────────────────────┐
│                                                                          │
│   GazeForegroundService                                                  │
│    ├── Camera2 API ──► MediaPipe Face/Hand Landmarker                   │
│    │                        └──► L2CS-Net ONNX (gaze pitch/yaw)        │
│    ├── SensorEventListener ──► GravityRollDecoupler                     │
│    │                              (head rotation ≠ gaze intent)        │
│    ├── CropStabilizer (EMA + affine alignment)                          │
│    ├── ConfidenceEngine (visibility + eyelid + gyro)                   │
│    ├── BlinkDetector (EAR + blendshapes)                                │
│    ├── DirectionHysteresis (entry/exit thresholds)                      │
│    ├── GazeStateMachine                                                  │
│    │    NO_FACE → SEARCHING → TRACKING → STABLE_LOCK → ACTION_READY    │
│    ├── InteractionProfileManager (per-app scroll tuning)                │
│    ├── ThermalPerformanceManager (dynamic frame throttling)              │
│    └── AccessibilitySafetyEngine                                        │
│                    └──► GazeAccessibilityService ──► System Gestures    │
│                                                                          │
└──────────────────────────────────────────────────────────────────────── ┘
Key Subsystems

GazeForegroundService.kt — The system heartbeat. Sticky foreground service that coordinates the full perception pipeline from frame capture to gesture dispatch.

L2CS-Net ONNX — Predicts high-resolution pitch/yaw gaze angles from MediaPipe eye ROI crops. Runs via OnnxReflectiveRunner with NNAPI hardware acceleration.

GravityRollDecoupler — Fuses accelerometer and gyroscope vectors to separate head rotation from genuine gaze intent. Essential for real-world usability when the device is not held perfectly upright.

GazeStateMachine — Prevents false positive gesture dispatch by enforcing the full state cycle: NO_FACE → SEARCHING → TRACKING → UNSTABLE → STABLE_LOCK → ACTION_READY → COOLDOWN. Actions only fire from ACTION_READY.

ConfidenceEngine — Combines landmark visibility scores, eyelid closure blendshapes, and device motion to generate a real-time 0.0–1.0 tracking quality score. Suppresses the filter chain during low-confidence periods.


Getting Started

Prerequisites
Requirement Version
Flutter SDK ^3.11.5
Dart SDK ^3.11.5
Android Studio Latest stable
Android SDK Platform Tools 33+
Physical Android Device API 29+ with front camera

⚠️ Emulator Note: Camera2 is unavailable on emulators. The app will launch but fall back to a degraded heuristic engine. Physical device testing is strongly recommended.

Installation
# 1. Clone the repository
git clone (https://github.com/Deep-SkyLabs/gaze-control.git
cd gaze-control

# 2. Install Flutter dependencies
flutter pub get

# 3. Run code generators (Riverpod, GoRouter)
flutter pub run build_runner build --delete-conflicting-outputs
Running in Development
# Connect your Android device and verify it's visible
flutter devices

# Run in debug mode (note: CV pipeline is noticeably slower in debug)
flutter run

# Run in profile mode for realistic performance with debugging
flutter run --profile
Building a Release APK
# Build optimized release APK
flutter build apk --release

# The output APK will be at:
# build/app/outputs/flutter-apk/app-release.apk

For App Bundle (recommended for Play Store distribution):

flutter build appbundle --release

Project Structure

gaze-control/
├── android/
│   └── app/src/main/kotlin/com/example/gaze/
│       ├── MainActivity.kt                  # Flutter ↔ Native MethodChannel bindings
│       ├── GazeAccessibilityService.kt      # Accessibility Service gesture dispatcher
│       └── GazeForegroundService.kt         # Full CV engine: camera, sensors, ML, state machine
├── lib/
│   ├── main.dart                            # App entry point
│   └── src/
│       ├── app/
│       │   └── app.dart                     # Root widget, theme injection
│       ├── core/
│       │   ├── platform/                    # MethodChannel / EventChannel adapters
│       │   ├── services/                    # Foreground service lifecycle controllers
│       │   └── theme/                       # Forui design token customization
│       ├── routing/
│       │   └── app_router_config.dart       # GoRouter declarations
│       └── features/
│           ├── dashboard/                   # Real-time telemetry & state visualization
│           ├── gaze_tracking/               # Calibration wizard & diagnostic overlays
│           ├── onboarding/                  # First-launch welcome flow
│           ├── permissions/                 # Accessibility + overlay permission checks
│           └── settings/                   # Threshold configuration & app whitelisting
├── docs/
│   ├── assets/                             # Banner, demo GIF, screenshots
│   ├── ARCHITECTURE.md                     # Deep-dive architecture documentation
│   └── CALIBRATION.md                      # Calibration system explained
├── test/
│   ├── unit/                               # Dart unit tests
│   └── widget/                             # Flutter widget tests
├── CHANGELOG.md
├── CONTRIBUTING.md
├── LICENSE
└── pubspec.yaml

Configuration Reference

All user configuration persists in Android SharedPreferences:

Key Type Default Description
flutter.user_calibration_profile String (JSON) null Neutral gaze vectors, directional bounds, deadzones, and biases from the 9-stage calibration flow
is_first_launch bool true Controls whether the onboarding carousel renders on next launch
sensitivity double 0.5 Global gesture sensitivity multiplier [0.1 – 1.0]

Calibration profiles use the following JSON schema:

{
  "neutralPitch": 0.0,
  "neutralYaw": 0.0,
  "topThreshold": 0.18,
  "bottomThreshold": -0.18,
  "leftThreshold": -0.22,
  "rightThreshold": 0.22,
  "deadzonePitch": 0.06,
  "deadzoneYaw": 0.06,
  "verticalBias": 1.0,
  "horizontalBias": 1.0
}

Contributing

Contributions are welcome. Please read CONTRIBUTING.md before opening a pull request.

Quick Contribution Guidelines
  • Design tokens only: All colors, spacing, and typography must use context.theme properties. No raw hex values.
  • Explicit state: Use NotifierProvider implementations. Avoid implicit provider reads in business logic.
  • Lint compliance: Run flutter analyze before submitting. The project uses strict linting rules in analysis_options.yaml.
  • Tests: Widget tests for new feature screens; unit tests for any new business logic in the core/ layer.
Good First Issues

Look for issues tagged good first issue — these are scoped, well-documented, and don't require deep knowledge of the native CV pipeline.


Roadmap

  • [ ] Smart App Detection — Auto-wake camera service when a supported app enters foreground
  • [ ] Context-Aware Smoothing — Dynamic One Euro Filter coefficients based on active scroll velocity
  • [ ] Ambient Dynamic Theming — Mirror the active app's color palette in the Gaze Control overlay
  • [ ] iOS Port — ARKit-based gaze engine for iPhone
  • [ ] Passive Calibration Refinement — Incremental profile tuning over time without explicit recalibration sessions
  • [ ] Additional Gesture Vocabulary — Tongue detection, expression triggers, breath sensing

Track progress and vote on priorities in Discussions → Roadmap.


Privacy

Gaze Control is designed privacy-first by architecture:

  • Camera frames are processed in native memory and immediately discarded — they are never written to disk
  • No gaze coordinates, usage events, or identifiers are transmitted off-device
  • The only persistent data is your calibration profile and settings, stored locally in Android SharedPreferences
  • The app requests no internet permission

License

Apache License 2.0 — see LICENSE for full terms.


Built with care by Deep Skylabs

Website · Blog

If Gaze Control has been useful to you, consider starring the repo — it helps others find it.

⚠️ Early Development — This project is actively in development. Some features may be incomplete, unstable, or subject to breaking changes. Use in production at your own discretion.

Deep Skylabs GitHub avatar
GitHub लेखक

Deep Skylabs ↗

A modern software engineering and AI infrastructure company focused on building practical products, developer tools, automation systems, and scalable digital ex

भाषाएँKotlinDartC++CMakeSwiftHTMLCObjective-C

श्रेणियाँ

उपयोग की गई निर्भरताएँ

निर्भरता सूची 13 आइटम

ऐप श्रेणियाँ

अपने प्रोजेक्ट के लिए रिपॉज़िटरी कैसे चुनें?

उपयोग के अनुसार छाँटें, गतिविधि की तुलना करें और अपनाने से पहले मूल प्रोजेक्ट की आवश्यकताएँ जाँचें।

  1. नाम, लेखक या विवरण से खोजें और तकनीकी विषय से परिणाम सीमित करें।
  2. स्टार, फ़ोर्क और हाल के पुश की तुलना करें। स्टार रुचि दर्शाते हैं, गुणवत्ता की गारंटी नहीं।
  3. उपयोग से पहले GitHub पर README, लाइसेंस और Issues जाँचें।

स्रोत और रखरखाव

“A repository is the most basic element of GitHub.”

GitHub Docs — About repositories ↗