Presentation

· Contributors · Organizations · Search Program · Flagged · My Recommendations · Happening Now

CANNON: A CXL-Based Near-Memory Processing Architecture for Approximate Nearest Neighbor Search on Real Hardware

SessionIntelligent Fetch & Match Architectures

DescriptionApproximate Nearest Neighbor (ANN) search is a foundational primitive for AI applications such as Retrieval-Augmented Generation (RAG). CPU and GPU-based solutions face scalability bottlenecks due to limited local memory, while a multi-tier architecture using SSDs introduces high latency from coarse-grained I/O, mismatched with fine-grained data access patterns inherent to ANN search. We present CANNON (A CXL-Based Near-Memory Processing Architecture for Approximate Nearest Neighbor Search on Real Hardware), a fully offloaded Near-Memory Processing (NMP) architecture implemented on real CXL hardware. CANNON transforms the ANN search pipeline into a fine-grained, deeply pipelined dataflow architecture to maximize throughput, and introduces asynchronous hashing, a speculative execution mechanism that hides hash-check latency to prevent pipeline stalls. Evaluated on large-scale vector datasets, CANNON achieves up to two orders of magnitude performance improvement over state-of-the-art CPU and GPU baselines.

Authors