Presentation
Freebit: Unleashing the Performance Potential of Low-Bit LLMs Through PIM
DescriptionLow-bit large language models (LLMs) use quantization to compress weights to 1/2/4 bits, significantly shrinking model size while preserving accuracy. Existing work leverages the reduced precision to cut computation. However, roofline analysis on the NVIDIA A100 shows limited inference speedup even with a 6.7× reduction in computation.This limitation is caused by insufficient memory bandwidth. Processing-in-memory (PIM) provides a promising solution for the memory bottleneck by integrating compute near data. To support efficient low-bit LLM inference, PIM should be carefully designed with joint software and hardware optimizations.
This paper presents FreeBit, a PIM-based architecture that unleashes the performance potential of low-bit LLMs. The objective is to capture the low-bit nature to better exploit PIM through hardware-software co-design. At the hardware level, a lookup-table (LUT)-centric architecture is designed to support quantized computation and minimize redundant computation. A sparsity-aware memory optimization is introduced to optimize memory access and leverage PIM bandwidth. At the software level, a static-dynamic decoupled scheduling strategy is presented to exploit PIM parallelism. Experimental results show that FreeBit effectively reduces redundant computation and memory access, and delivers notable performance improvements over CPUs, GPUs, and prior PIM baselines.
This paper presents FreeBit, a PIM-based architecture that unleashes the performance potential of low-bit LLMs. The objective is to capture the low-bit nature to better exploit PIM through hardware-software co-design. At the hardware level, a lookup-table (LUT)-centric architecture is designed to support quantized computation and minimize redundant computation. A sparsity-aware memory optimization is introduced to optimize memory access and leverage PIM bandwidth. At the software level, a static-dynamic decoupled scheduling strategy is presented to exploit PIM parallelism. Experimental results show that FreeBit effectively reduces redundant computation and memory access, and delivers notable performance improvements over CPUs, GPUs, and prior PIM baselines.
Event Type
Research Manuscript
TimeWednesday, July 291:30pm - 1:42pm PDT
LocationMtg Room 203AB