Presentation
Fusedot: A Multiplication-Fused Dot Product Accelerator for Efficient LLM Inference
DescriptionWeight quantization enables efficient LLM inference but requires mixed-precision computation (e.g., FP16xINT4), which is not efficient in general-purpose processors. Existing accelerators integrate custom mixed-precision arithmetic units but follow the dot-product paradigm, which suffers from numerous multiplications. This work exploits multiplication fusion in low-bit quantization and introduces MFDP, a novel dot-product paradigm that fuses identical-weight multiplications into a single operation. Based on MFDP, we propose FuseDot, an accelerator that optimizes the fusion process by incorporating index-driven generalized matrix multiplication and a hierarchical dual-phase reordering algorithm. Additionally, a cross-PE multiplier-sharing architecture with a half-multiplier scheme further amortizes multiplier costs across multiple PEs. This work achieves 1.51–1.98x speedup and 1.19–1.51x higher energy efficiency over state-of-the-art accelerators.
Event Type
Research Manuscript
TimeMonday, July 2710:30am - 10:43am PDT
LocationMtg Room 101B