Presentation
PEACE: Algorithm–hardware Co-Design for PE-Granular Energy-Aware Compressed GEMM Acceleration
DescriptionGeneral matrix–matrix multiplication (GEMM) remains the dominant compute kernel in transformer-based large language model (LLM) inference.
However, large-scale matrices exhibiting irregular sparsity fundamentally limit throughput and energy efficiency in existing systems.
To address these challenges, we propose PEACE, an energy-aware matrix compression framework for accelerating GEMM operations at the processing element (PE)-array granularity.
PEACE comprises a novel matrix compression algorithm, PEACE-Alg, which reorganizes large-scale matrices into hardware-friendly compressed formats by exploiting the column-wise energy distribution.
To support PEACE-Alg, we design a hardware microarchitecture, PEACE-Hw, that integrates RISC-V ISA extensions with a table-metadata fetcher and a metadata-driven operand loader to feed a reconfigurable systolic PE array, and a dedicated partial-sum adder for efficiently merging intermediate results.
Experimental results show that PEACE occupies 2.705 mm^2 in a 14 nm ASIC and delivers a peak INT8 throughput of 88 GOPS/W.
PEACE achieves 1.60×-1.67× speedup over a RISC-V core baseline across 14 transformer-based LLMs.
Compared with state-of-the-art designs, PEACE provides 4.4× higher PE density, 1.79× average speedup and 2.53× energy efficiency.
However, large-scale matrices exhibiting irregular sparsity fundamentally limit throughput and energy efficiency in existing systems.
To address these challenges, we propose PEACE, an energy-aware matrix compression framework for accelerating GEMM operations at the processing element (PE)-array granularity.
PEACE comprises a novel matrix compression algorithm, PEACE-Alg, which reorganizes large-scale matrices into hardware-friendly compressed formats by exploiting the column-wise energy distribution.
To support PEACE-Alg, we design a hardware microarchitecture, PEACE-Hw, that integrates RISC-V ISA extensions with a table-metadata fetcher and a metadata-driven operand loader to feed a reconfigurable systolic PE array, and a dedicated partial-sum adder for efficiently merging intermediate results.
Experimental results show that PEACE occupies 2.705 mm^2 in a 14 nm ASIC and delivers a peak INT8 throughput of 88 GOPS/W.
PEACE achieves 1.60×-1.67× speedup over a RISC-V core baseline across 14 transformer-based LLMs.
Compared with state-of-the-art designs, PEACE provides 4.4× higher PE density, 1.79× average speedup and 2.53× energy efficiency.
Event Type
Research Manuscript
TimeTuesday, July 284:18pm - 4:30pm PDT
LocationMtg Room 202C