Presentation
CAPA: Convertibility-Aware Mixed-Precision Accelerator for Asymmetric GEMM in Weight-Only Quantized LLMs
DescriptionFP-INT GEMM accelerators for weight-only quantized large language models suffer from large partial sum (psum) overhead and a lack of precision-convertibility, limiting scalability. To mitigate this, we propose CAPA, an accelerator achieving high efficiency and versatility. (1) CAPA introduces Hybrid Delta Block Floating Point (HDBFP), a novel INT-based format that reduces psum overhead, preserves accuracy, and integrates BF16/FP16 arithmetic. (2) CAPA adopts Weight Decomposition to unify INT8 and INT4 arithmetic, enabling parallel processing. We demonstrate CAPA attains identical GPU accuracy, improving area efficiency by 3.98x and power efficiency by 4.38x over the FP-FP baseline.
Event Type
Research Manuscript
TimeTuesday, July 2811:18am - 11:30am PDT
LocationMtg Room 101A
Similar Presentations