Presentation
Terastal: Layer-Variant-Based Scheduling for Real-Time Multi-DNN Workloads on Heterogeneous Accelerators
DescriptionHeterogeneous DNN accelerators map layers in real-time multi-DNN workloads to preferred accelerators to minimize latency. However, under skewed workloads, cross-accelerator layer latency difference causes deadline misses. We introduce layer variants, customized layer implementations to reduce latency on non-preferred accelerators. We then present Terastal, a real-time layer-variant design and scheduling framework for heterogeneous DNN accelerators. Terastal uses efficient network architectures to design variants, performs virtual budget assignment to guide variant design and scheduling, and dynamically applies variants online to minimize deadline violation and accuracy loss. Terastal reduces deadline miss rate per model by 34.1% versus DREAM scheduler, with minimal accuracy loss.
Event Type
Work in Progress
TimeMonday, July 276:59pm - 7:00pm PDT
LocationExhibit Hall
Similar Presentations