Presentation
LOHA: A Latency-Optimized CPU-Storage Hybrid Architecture for Billion-Scale Graph-Based Vector Similarity Search
DescriptionGraph-based Approximate Nearest Neighbor Search (GANNS) has been widely adopted for vector retrieval owing to its high scalability and low latency. However, existing GANNS frameworks face severe latency bottlenecks on billion-scale datasets due to excessive SSD I/O. This work presents LOHA, a latency-optimized CPU–storage hybrid architecture for product quantization (PQ)-based billion-scale ANNS. LOHA decouples graph traversal and re-ranking workloads, executing the former on the CPU while offloading the latter to an in-storage accelerator that exploits SSD-level parallelism. Furthermore, a speculative re-ranking mechanism pipelines both stages to minimize idle time and reduce end-to-end latency. Experiments show that LOHA achieves throughput improvements of up to 9.6x, 6.2x, and 4.0x over CPU-, GPU-, and in-storage architectures, respectively.
Event Type
Research Manuscript
TimeMonday, July 272:08pm - 2:21pm PDT
LocationMtg Room 202C
Similar Presentations