Presentation

· Contributors · Organizations · Search Program · Flagged · My Recommendations · Happening Now

Hiband: A Hierarchical Bandwidth-Aware HB-DRAM Accelerator with Array-Level Co-Execution and Reconfigurable NoC for LLM Inference

SessionEfficient LLM and MoE Inference on Specialized Hardware

DescriptionHybrid-bonding 3D DRAM (HB-DRAM) delivers massive in-situ bandwidth but shifts LLM accelerator bottlenecks from off-chip memory to the Network-on-Chip, where decentralized bank-local bandwidth must be coordinated across arrays. We present HiBand, a bandwidth-aware HB-DRAM accelerator that co-designs array-level execution with a four-mode reconfigurable NoC. HiBand combines grouped tensor parallelism and cross-head array-level co-execution to confine most traffic to short-range links while overlapping bandwidth-bound attention with compute-bound feed-forward layers. On LLM models mapped to a 32-array HB-DRAM accelerator, HiBand achieves up to 4.28× speedup over an HBM3 GPU-style accelerator and 1.67× speedup over a state-of-the-art HB-DRAM–based NMP design.

Authors