Presentation
NASiC: 3D NAND-Based CAM-Selected Multibit CIM Architecture for Efficient On-Device Mixture-of-Experts LLM Inference
DescriptionThe Mixture-of-Experts (MoE) models have emerged as the state-of-the-art paradigm for scaling up large language models (LLMs) without a proportional increase in computational cost. However, on-device deployment of MoE models still faces a critical challenge due to the large memory requirement for storing all expert parameters. In this work, we proposed NASiC, a 3D NAND-based CAM-selected multibit CIM architecture through algorithm-hardware co-optimization, tailored to the high-density storage and sparse computation requirements of MoE models. V-ASIC architecture achieves improved throughput, high area- and energy-efficiency, indicating its great potential for on-device MoE inference.
Event Type
Research Manuscript
TimeMonday, July 2711:23am - 11:36am PDT
LocationMtg Room 203AB
Similar Presentations