Presentation
PIMGRAG: A Heterogeneous PIM Architecture for Graph-Based Retrieval-Augmented Generation
DescriptionGraph-based retrieval-augmented generation (RAG) improves the interpretability and factual consistency of large language models (LLMs) through structured knowledge graphs. Despite these benefits, graph-based RAG suffers from inefficient retrieval. The retrieval stage causes massive movement of vector and graph data between memory and processors. This movement leads to low arithmetic intensity and heavy pressure on memory bandwidth.
This work presents PIMGRAG, a heterogeneous architecture that accelerates graph-based RAG through hardware/software co-design.
At the hardware level, PIMGRAG designs a PIM architecture for the retrieval stage, which reduces off-chip data transfer by executing bandwidth-intensive operations near memory.
At the software level, PIMGRAG applies a lightweight scheduling method that orchestrates PIM and GPU execution and lowers idle time across stages.
Evaluation results show improvements in throughput, latency, and energy efficiency over CPU–GPU and existing PIM-based baselines.
This work presents PIMGRAG, a heterogeneous architecture that accelerates graph-based RAG through hardware/software co-design.
At the hardware level, PIMGRAG designs a PIM architecture for the retrieval stage, which reduces off-chip data transfer by executing bandwidth-intensive operations near memory.
At the software level, PIMGRAG applies a lightweight scheduling method that orchestrates PIM and GPU execution and lowers idle time across stages.
Evaluation results show improvements in throughput, latency, and energy efficiency over CPU–GPU and existing PIM-based baselines.
Event Type
Research Manuscript
TimeMonday, July 272:47pm - 3:00pm PDT
LocationMtg Room 202C