Presentation
RAGNMP: Leveraging Elimination Tree to Accelerate RAG with Near-Memory Processing
DescriptionRetrieval-Augmented Generation (RAG) enhances Large Language Models (LLMs) by integrating information retrieval from knowledge databases, significantly improving the generated results' accuracy, relevance, and contextual richness. An in-depth analysis of RAG reveals that its diverse operations are primarily constrained by memory bottlenecks. The diversity and continual evolution of RAG algorithms further increase system design complexity. In this paper, we introduce RAGNMP, a general-purpose Near-Memory Processing (NMP) accelerator designed for RAG. Specifically, we first propose an enhanced and quantified elimination tree variant that simultaneously explores data placement, task parallelism, and pipelining to better support RAG workloads on NMP architectures. It also remains adaptable to algorithm changes in RAG. We further propose a general-purpose NMP architecture with a flexible processing unit that efficiently supports diverse memory-bound operations in RAG. Experimental results show that RAGNMP outperforms the state-of-the-art RAG system and accelerator.
Event Type
Research Manuscript
TimeMonday, July 271:55pm - 2:08pm PDT
LocationMtg Room 202C