Semantic views reduce costly ai model calls while ensuring accuracy
Semantic Views: Certified Reuse of LLM Labels Across Semantic Queries
DatabasesArtificial Intelligence
Summary
Answering questions using large language models is expensive because each question requires the model to work hard. The authors introduce SemViews, a system that smartly reuses answers from earlier questions to save these costly model calls without losing accuracy. It orders possible answers based on cheaper signals from previous queries and only confirms uncertain cases with the expensive model. Tests show SemViews saves lots of these costly checks while still being reliable almost all the time.
What this means in practice
- •For data platform teams: Reduce the number of large language model calls in semantic search systems by reusing past query labels with guaranteed accuracy.
- •For digital content managers: Lower costs of natural-language filtering in large document catalogs by caching and certifying earlier model label results for repeated searches.
Authors
Oktie Hassanzadeh, Dharmashankar Subramanian
Abstract
Semantic query engines evaluate natural-language filters with a large language model, and every evaluation is a costly oracle call. Optimizers that reduce these calls face a dilemma: cascades that trust a cheap proxy miss their precision and recall targets on many queries, methods with finite-sample guarantees save little, and all of them discard the oracle's labels when the query ends. We present SemViews, a certified semantic filter that orders tuples by signals that cost no oracle calls (the labels that earlier queries left behind as semantic views, a local embedding, and a helper model's score), refines that order with labels it must pay for anyway, and certifies its answer with fresh stratified oracle samples. The answer meets user precision and recall targets with probability at least $1-δ$ for every workload and every catalog, so a stale or misleading view can cost oracle calls but never accuracy. On four workloads replayed against 725,000 recorded oracle labels, SemViews meets both targets on 99.63% of queries and saves 28%-45% of the oracle calls, whereas the only prior method in our study with the same guarantee needs 1.32x-1.71x as many calls. When a question returns in different words, views cut its calls by a further 27%-36%, with both targets met on at least 99.8% of such queries; similarity caches and entailment-based reuse meet them on only 17%-47%.