Presentation
Optimizing Dynamic-Shape Neural Networks: A Unified Approach to Adaptive Tuning
DescriptionDynamic-shape neural networks are essential for handling variable-length data in tasks, but their optimization is challenging. Existing methods, such as hand-tuned libraries or static kernel compilation, are inefficient due to a lack of flexibility or the high overhead from repeated compilations.
We propose DyTuner, an adaptive operator-level tuning framework for dynamic-shape neural networks. DyTuner uses a unified, system-compatible interface and an adaptive algorithm to efficiently tune kernels for varying shapes, significantly improving end-to-end performance and reducing redundant compilations. Experiments demonstrate that DyTuner effectively optimizes dynamic-shape workloads, achieving speedups of up to 1.41x and 1.64x over state-of-the-art solutions, BladeDISC and TorchInductor.
We propose DyTuner, an adaptive operator-level tuning framework for dynamic-shape neural networks. DyTuner uses a unified, system-compatible interface and an adaptive algorithm to efficiently tune kernels for varying shapes, significantly improving end-to-end performance and reducing redundant compilations. Experiments demonstrate that DyTuner effectively optimizes dynamic-shape workloads, achieving speedups of up to 1.41x and 1.64x over state-of-the-art solutions, BladeDISC and TorchInductor.
Event Type
Research Manuscript
TimeWednesday, July 295:16pm - 5:30pm PDT
LocationMtg Room 201B