Presentation
Late Breaking Results: Recoverability-Guided Layer-Wise N:M Sparsity Under Latency Constraints
DescriptionLayer-wise N:M sparsity balances accuracy and hardware acceleration for Vision Transformers (ViTs), yet identifying effective configurations is costly due to fine-tuning overhead and latency-induced fragmentation. We present HaLSpar, a hardware-aware framework that couples a recoverability-driven zero-shot proxy (RCG) with a latency-constrained search strategy. By estimating recovery potential without repeated fine-tuning, HaLSpar directly optimizes sparsity configurations. On ImageNet-1K with multiple ViT and Swin models, it achieves up to 270x faster search while delivering 1.5x–2.5x speedups with minimal accuracy degradation.
Event Type
Late Breaking Results
TimeMonday, July 275:03pm - 5:06pm PDT
LocationExhibit Hall
Similar Presentations