Semantic filter results reused safely to reduce redundant large language model calls
SemReuse: Certified Reuse of Cached Natural-Language Views for Semantic Operators
Databases
Summary
Filtering data using natural language usually means asking a language model for every piece of data, which can be slow and expensive. The authors built SemReuse, a system that remembers past filter results and safely reuses them when similar filters appear again. It carefully checks if reusing results is safe and audits the process to avoid mistakes, only asking the language model again when necessary. This makes filtering much faster while still being accurate.
What this means in practice
- •For data platform engineers: Avoid repeated language model calls on natural language filters by safely reusing cached filter results for faster analytical queries.
- •For content moderation teams: Reduce cost and latency when filtering large text datasets with repeated or related natural language conditions by caching and certifying reuse of filter outcomes.
Authors
Phat T. Tran-Truong, Tung Kieu, Xuan-Bach Le, Son X. Ha, Duc Kieu, Bin Yang, Christian S. Jensen
Abstract
Semantic operators allow users to filter unstructured data using conditions expressed in natural language, but existing systems typically invoke an LLM for every input tuple. Analytical workloads often contain repeated or related filters whose results could be reused. Such reuse is risky because it requires inferring whether filter conditions are equivalent, whether one implies another, or whether they are mutually exclusive. An incorrect judgment can omit qualifying tuples or include nonqualifying ones. We introduce SemReuse, a system for safely reusing previous semantic-filter results. SemReuse stores each result as a materialized view and compares new filters with existing views to identify tuples that can provisionally be accepted or skipped without another LLM call. Because these decisions rely on a fallible language reasoner, SemReuse audits every rewritten result. Its pooled stratified audit uses exact finite-population bounds to provide jointly valid per-query recall and precision guarantees. If the requested guarantees cannot be established, SemReuse evaluates additional tuples and progressively approaches full evaluation. Thus, incorrect relation judgments affect efficiency rather than the validity of the reported guarantees. Experiments on large labeled corpora and real LLM executions show reductions of up to 15.4x. They also identify cross-filter inconsistency as the main source of repair cost.