Presentation
PAGE: Processing-Using-DRAM Architecture-Circuits Co-Optimization for Efficient Acceleration of General Matrix-Vector Multiplication
DescriptionAs large language models (LLMs) continue to scale in size and complexity, DRAM access has become a performance bottleneck, especially during the memory-bound decoding phase dominated by general matrix-vector multiplication (GEMV) operations. Recent work has explored near-/in-memory computing to address bandwidth limitations. Among these, Processing-Using-DRAM (PUD) offers a unique advantage as it performs logic operations entirely within DRAM peripherals with minimal circuit changes, enabling deployment on commercial off-the-shelf (COTS) DRAM. While prior work reported multi-input logic operations, its potential to accelerate GEMV operations in LLMs remains underexplored due to the limited compute capability of DRAM sense amplifiers (SAs).
To overcome this challenge, we propose PAGE, a PUD architecture that accelerates GEMV through four key optimizations: (1) mat-level parallelism that exploits row-level parallelism by activating SAs across multiple mats; (2) inversion-enabled SA supporting NOT operations within a mat; (3) AP-based row copy mechanism that performs simultaneous source–destination activation with only control-signal changes; and (4) adaptive adder-tree accumulation that reduces accumulation cycles. We demonstrate the baseline PUD architecture on a COTS DRAM–FPGA platform to verify the functionality and timing of PUD operations, implement the modified circuits in TSMC 16nm for circuit-level evaluation, and build a in-house simulator for system-level throughput and energy analysis. Overall, PAGE achieves up to 14× and 3.1× end-to-end throughput and energy improvement for GEMV workloads over the SoTA PUD designs, demonstrating its feasibility for accelerating memory-bound LLM workloads with PUD.
To overcome this challenge, we propose PAGE, a PUD architecture that accelerates GEMV through four key optimizations: (1) mat-level parallelism that exploits row-level parallelism by activating SAs across multiple mats; (2) inversion-enabled SA supporting NOT operations within a mat; (3) AP-based row copy mechanism that performs simultaneous source–destination activation with only control-signal changes; and (4) adaptive adder-tree accumulation that reduces accumulation cycles. We demonstrate the baseline PUD architecture on a COTS DRAM–FPGA platform to verify the functionality and timing of PUD operations, implement the modified circuits in TSMC 16nm for circuit-level evaluation, and build a in-house simulator for system-level throughput and energy analysis. Overall, PAGE achieves up to 14× and 3.1× end-to-end throughput and energy improvement for GEMV workloads over the SoTA PUD designs, demonstrating its feasibility for accelerating memory-bound LLM workloads with PUD.
Event Type
Research Manuscript
TimeTuesday, July 2810:30am - 10:43am PDT
LocationMtg Room 203AB