Efficient and Effective In-place Graph-based Vector Index Updates
DatabasesInformation Retrieval
Summary
The authors address the challenge of keeping large language model data up-to-date while ensuring fast and accurate searches. They point out that existing methods either update data quickly but with worse search quality, or keep good search quality but slow updates. To tackle this, they created Yi, a system that updates vector data directly in place while balancing speed and accuracy. Yi uses a new method involving three parts: a small task-focused engine, an asynchronous buffer, and a vector file system. Their tests show Yi updates and searches faster than current systems while using less memory and CPU power.
Authors
Haotian Liu, Yujun He, Bo Tang
Abstract
In the era of Large Language Models (LLMs), efficient vector updates are critical for capturing real-time information from rapidly evolving data. However, it is not trivial to process frequent vector insert and delete updates and maintain a high recall of the search results simultaneously. Specifically, the cluster-based vector indexing methods have high update throughput but low search result quality. Existing out-of-place graph-based vector indexing update approaches suffer from poor update throughput due to the need to periodically merge update batches into the underlying graph index. Building a vector data system that supports efficient and effective in-place updates is inherently challenging. In this work, we propose Yi to achieve it. In particular, Yi supports in-place graph-based vector indexing updates with consistently high update throughput and good search result quality. The key idea of Yi is decomposition facilitates consolidation. In particular, we introduce a vector-level update mechanism and architect Yi with three core components: (i) a tasklet-based execution engine, (ii) an asynchronous buffer manager, and (iii) a vector file system. Experimental results demonstrate that Yi achieves 1.75x higher update throughput and 1.8x higher concurrent search throughput than the state-of-the-art systems on the 800M dataset, while using only 73% of the peak memory and fewer CPU cores.