When Knowledge Is Not Free: Cost-Aware Evidence Selection in Retrieval-Augmented Generation
2026-06-01 • Computation and Language
Computation and Language
AI summaryⓘ
The authors study a version of Retrieval-Augmented Generation (RAG) where accessing information can cost money, instead of being free. They create a setup where different information sources have different cost levels, and systems have to answer questions without exceeding a budget. Their tests show that simply picking fixed sets of information isn't reliable, and spending more money doesn't always help. They then explore smarter systems that decide when and what to look up based on costs, which work well but depend a lot on the specific model and task. The authors highlight that managing the cost of finding helpful info is an important challenge for improving these systems.
Retrieval-Augmented Generationaccess costevidence selectionbudgeted retrievalquestion answeringlarge language modelsadaptive retrievalMS MARCOdomain-specific QAcost-aware systems
Authors
Mingyan Wu, Han Yang, Omer Ben-Porat, Yftah Ziser
Abstract
Retrieval-Augmented Generation (RAG) typically assumes that external knowledge is free, but many high-quality sources are paywalled, licensed, restricted, or otherwise costly to access. We introduce cost-aware RAG, a setting where retrieved evidence is assigned access-cost tiers and systems must answer under an explicit evidence-access budget. We instantiate this setting by augmenting MS MARCO v2.1 with access-friction tiers and evaluate budgeted evidence selection across general-domain and domain-specific QA benchmarks. Our results show that static selection is brittle: no fixed selector uniformly dominates, and larger budgets do not reliably improve answer quality, even when costly evidence is domain-matched. We then study agentic cost-aware RAG, where an LLM decides when to retrieve, which tier to access, and when to stop. Agents show strong promise as adaptive evidence-acquisition controllers, but their behavior remains highly model- and task-dependent. These findings suggest that cost-aware evidence acquisition is a central challenge for the next generation of RAG systems. All code and data are available at https://github.com/Mignonmy/Cost-Aware.