NDSEARCH: Accelerating Graph-Traversal-Based Approximate Nearest Neighbor Search through Near Data Processing

Approximate nearest neighbor search (ANNS) is a key retrieval technique for vector database and many data center applications, such as person re-identification and recommendation systems. It is also fundamental to retrieval augmented generation (RAG) for large language models (LLM) now. Among all th...

Celý popis

Uložené v:
Podrobná bibliografia
Vydané v:2024 ACM/IEEE 51st Annual International Symposium on Computer Architecture (ISCA) s. 368 - 381
Hlavní autori: Wang, Yitu, Li, Shiyu, Zheng, Qilin, Song, Linghao, Li, Zongwang, Chang, Andrew, Li, Hai lHelenr, Chen, Yiran
Médium: Konferenčný príspevok..
Jazyk:English
Vydavateľské údaje: IEEE 29.06.2024
Predmet:
Approximate Nearest Neighbor Search
Bandwidth
Computational modeling
Graphics processing units
Hardware/Software Co-Design
Memory management
Near Data Processing
Nearest neighbor methods
Parallel processing
Throughput
On-line prístup:Získať plný text
Tagy: Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
Popis
Shrnutí:Approximate nearest neighbor search (ANNS) is a key retrieval technique for vector database and many data center applications, such as person re-identification and recommendation systems. It is also fundamental to retrieval augmented generation (RAG) for large language models (LLM) now. Among all the ANNS algorithms, graph-traversal-based ANNS achieves the highest recall rate. However, as the size of dataset increases, the graph may require hundreds of gigabytes of memory, exceeding the main memory capacity of a single workstation node. Although we can do partitioning and use solid-state drive (SSD) as the backing storage, the limited SSD I/O bandwidth severely degrades the performance of the system. To address this challenge, we present NDSEARCh, a hardware-software co-designed near-data processing (NDP) solution for ANNS processing. NDSeARCH consists of a novel in-storage computing architecture, namely, SEARSSD, that supports the ANNS kernels and leverages logic unit (LUN)-level parallelism inside the NAND flash chips. NDSEARCH also includes a processing model that is customized for NDP and cooperates with SearSSD. The processing model enables us to apply a two-level scheduling to improve the data locality and exploit the internal bandwidth in NDSearch, and a speculative searching mechanism to further accelerate the ANNS workload. Our results show that NDSEARCH improves the throughput by up to 31.7 \times, 14.6 \times, 7.4 \times 2.9 \times over CPU, GPU, a state-of-the-art SmartSSD-only design, and DeepStore, respectively. NDSEARCH also achieves two orders-of-magnitude higher energy efficiency than CPU and GPU.
DOI:10.1109/ISCA59077.2024.00035