Retrieval augmented generation better triages query source risks
Query-Aware Source-Risk Triage for Retrieval-Augmented Generation
Artificial Intelligence
Summary
Sometimes when computer systems answer questions by looking up information online, they might miss how the sources relate to the question. This paper studies a new method that checks how each retrieved page matches the type of question and treats these connections carefully before generating an answer. The method groups similar queries, scores pages in multiple ways, and decides whether to pass, explain, exclude, or review each page for accuracy. The authors tested their approach on real and simulated data to understand how well it works and how to improve future automated checks.
What this means in practice
- •For search engine teams: Introduce query-dependent filtering to reduce low-quality or irrelevant source content before answer generation.
- •For content moderation teams: Use triage scoring to prioritize reviewing online sources that could affect automated content summaries.
Tested on simulated data.
Authors
Kainan Zhou, Gangzhen Qian, Chuhong Xu, Lu Yi
Abstract
Retrieval-augmented generation (RAG) pipelines may omit a source's material relationship to the query. We study a pre-generation triage layer that treats this relationship as query dependent. The method routes canonical query families for enhanced review and assigns retrieved pages to pass, contextualize, exclude, or review. It combines a four-dimension page score, rank-discounted family aggregation, intent-preserving query mutations, and a family-held-out router. A single-coded pilot of 200 real URLs supplies provisional calibration anchors; a 20,000-row scenario with synthetic domain identifiers supports controlled workload analysis. An oracle page gate defines a risk-coverage target for a future learned classifier. The evaluation shows why page-level frequency cannot substitute for family-level exposure and quantifies how calibration changes scenario activation. Annotation reliability remains unmeasured, and synthetic rankings omit real retrieval dynamics. The result is an auditable triage method and validation plan, not an estimate of deployed review workload, live-Web prevalence, or downstream answer-quality gains.