VADER speeds up filtered vector searches with automatic recall control
VADER: Filtered Vector Search with Declarative Recall
Databases
Summary
Filtered vector search helps computers find related items based on both data details and numeric patterns, which is usually complicated and needs a lot of fine-tuning. The authors present VADER, a system that lets users set how complete they want their search results to be, and then automatically adjusts the search process to meet that goal without manual tweaking. VADER predicts how well the search is doing and stops early when the target is reached, making searches faster and better. Tests show that VADER is often quicker and provides higher quality results than previous methods.
What this means in practice
- •For enterprise data engineers: Enable fast and efficient searches over large datasets combining structured data and vector embeddings while meeting specific recall accuracy without manual tuning.
- •For search infrastructure teams: Improve recommendation systems by dynamically controlling search completeness, ensuring faster responses with consistent result quality under varying filter conditions.
- •For e-commerce platform developers: Offer product recommendations filtered by user preferences with guaranteed result completeness and improved speed using automatic recall targeting.$Commercial implications: Supports building better personalized search and recommendation features that adapt to user filters, enhancing user experience and sales.
Authors
Manos Chatzakis, Duo Lu, Helena Caminal, Yannis Chronis, Fatma Özcan, Yannis Papakonstantinou, Timofey Asyrkin, Sebastian Infante Murguia, Itai Rosenblatt, Themis Palpanas
Abstract
Approximate filtered vector search (FVS), a core operation in many data management tasks that combine structured data with vector embeddings, exhibits increased complexity due to the characteristics of filtering predicates. Each predicate is defined by selectivity (i.e., the fraction of vectors that satisfy the predicate) and correlation (i.e., the relationship between the filter and the vector space), which can significantly affect search difficulty even for the same query vector. This poses a key challenge for users aiming to integrate vector search with structured data, as efficient execution often requires extensive manual tuning of algorithm parameters. In this paper, we present VADER, the first approach that eliminates hyperparameter tuning by introducing declarative recall for approximate filtered vector search. With declarative recall, users specify a desired recall target, and VADER executes FVS queries to meet this target without requiring manual configuration. VADER achieves this by employing a filter-aware recall predictor that generalizes across varying selectivities and correlations without explicit tuning, and by performing early termination once the predicted recall reaches the user-defined target. Through extensive experimental evaluation, we show that VADER achieves near-optimal early termination, while providing significant speedups of up to 53% faster and improved result quality of 28% compared to the best-performing baseline.