Presentation
Late Breaking Results: Hybrid-Bonding Architecture with Stage-Aware Mapping for Masked Autoregressive Model Inference
DescriptionMasked autoregressive (MAR) models have recently shown strong potential in visual generation tasks. Unlike traditional autoregressive model that generates tokens sequentially, MAR predicts multiple tokens in parallel at each generation step (stage), resulting in non-linear growth in computation and memory across stages. To meet the demands, we introduce Hybrid-bonding (HB) architectures to provide high memory bandwidth and scalable compute capability through 3D integration of DRAM and logic banks. However, existing static mapping strategies cannot effectively adapt to the stage-wise workload shifts due to limited remote-bank bandwidth and constrained per-bank compute capacity. In this work, we propose a \textbf{stage-aware mapping strategy} for MAR inference on HB-based architectures that dynamically balances communication and computation costs across stages. Experimental results show up to \textbf{1.75$\times$ speedup} and \textbf{1.56$\times$ energy efficiency} compared with baselines.
Event Type
Late Breaking Results
TimeMonday, July 275:16pm - 5:19pm PDT
LocationExhibit Hall
Similar Presentations