ANNLib: A Development Framework for Efficient Approximate Nearest Neighbor Search
2026-07-20 • Machine Learning
Machine LearningInformation Retrieval
AI summaryⓘ
The authors developed ANNLib, a new software library to make finding approximate nearest neighbors (ANNS) in large data sets easier and faster. They separated the main parts of ANNS systems (algorithms and data structures) so each can be improved and swapped independently. Their library includes top existing methods and new ideas, letting users mix and match features like fast searching and dynamic updates. Tests show ANNLib works well across different tasks and is simple to use compared to earlier tools.
Approximate Nearest Neighbor SearchGraph-based algorithmsData structuresDynamic updatesFilter searchDeep learningSoftware libraryAlgorithm optimizationSnapshot queriesPerformance benchmarking
Authors
Zheqi Shen, Jingbo Su, Zijin Wan, Yan Gu, Yihan Sun
Abstract
Approximate Nearest Neighbor Search (ANNS) plays a pivotal role in modern deep learning pipelines. Recently, many ANNS systems have been proposed to either provide broad functionality or reach high performance. However, it is yet difficult to achieve both with minimal programming efforts. We propose ANNLib to address the gap. ANNLib is a library that provides a programming framework for achieving high performance and flexible functionality in ANNS systems, based on popular graph-based ANNS algorithms. We carefully decouple and independently optimize both the algorithm and the data structure components of an ANNS system. In addition, we integrate state-of-the-art algorithms and data structures into ANNLib as modules, along with our new designs. Users can choose combinations of components to implement sophisticated settings with high performance, such as filter search, fully dynamic updates, and historical queries on snapshots. Our experiments show that our new solution provides a simple interface for various applications and achieves comparable or even better performance than previous work, specifically for each application.