Presentation
Scale-Gest: Scalable Model-Space Synthesis and Runtime Selection for On-Device Gesture Detection
DescriptionRealizing on-device ML-based gesture detection under tight real-time performance, energy and memory constraints is challenging, especially when considering mobile devices with varying battery-power levels. Existing EdgeAI deployments typically rely on a single fixed detector, limiting optimization opportunities. We present Scale-Gest, a novel runtime-adaptive gesture detection framework that expands the detector space into a dense family of tiny-YOLO architectures. We introduce multiple novel device-calibrated ACE (Accuracy-Complexity-Energy) profiles by analyzing different model-resolution-stride operating points. A run-time controller selects an appropriate ACE mode under user-defined and battery-constraints, while a motion-aware hand-gesture-tracking ROI gate crops the input for reduced-complexity detection. To evaluate performance of our system in real-world car driving scenarios, we introduce a temporally-annotated Driver Simulated Gesture (DSG-18) dataset. Scale-Gest maintains event-level F1 while significantly reducing energy and latency compared to single-detector approaches. On a battery-powered laptop running gesture streams, our ACE controller reduces per-frame energy by ≈4× (from ≈6.9 mJ to ≈1.6 mJ) while maintaining high gesture-detection performance (event-level F1 ≈ 0.8–0.9) and low mean latency (≈6 ms).
Event Type
Research Manuscript
TimeWednesday, July 292:21pm - 2:34pm PDT
LocationMtg Room 101A
Similar Presentations