Presentation
DAP: A Software/hardware Co-Design for Accelerating Multi-Chip-Based Distributed Sparse-Dense Matrix Multiplication
DescriptionSparse-dense matrix multiplication (SpMM) is a core component in many critical applications such as deep learning and scientific computing. Existing SpMM accelerators employ the COO format to compress sparse matrices and rely on high-bandwidth off-chip memory for performance gains. However, four major challenges remain unaddressed: 1) Fixed 2-D partitioning strategies lead to workload imbalance among processing nodes due to irregular distribution of non-zero elements in sparse matrices. 2) Redundant metadata from the COO format results in a communication bottleneck that hinders the scalability of existing SpMM accelerators. 3) To accommodate irregular memory access patterns, the use of multiple data replicas significantly increases the pressure on on-chip storage resources. 4) Handling RAW hazards from floating-point adders in software incurs substantial pre-processing overhead. To address these challenges, we propose DAP, the first 2-D multi-chip-based architecture composed of dedicated accelerators, and SPU, a novel communication-friendly SpMM accelerator. To mitigate load imbalance caused by irregular non-zero distribution, we design a two-level matrix partitioning framework that effectively balances workloads across nodes in a 2-D computing array, achieving a performance improvement of 1.43x. Furthermore, the SPU adopts CSC over COO format to reduce communication traffic. SPU also minimizes redundant on-chip storage from data replication, with only a 0.067x performance penalty due to memory access conflicts. By employing a reservation buffer, the SPU resolves RAW dependencies without time-consuming pre-processing. Our simulation-based evaluation demonstrates DAP achieves geometric mean throughputs of 2.69x relative to NVIDIA A100 GPU at 500MHz frequency.
Event Type
Research Manuscript
TimeMonday, July 273:42pm - 3:54pm PDT
LocationMtg Room 203AB
Similar Presentations