Adaptive selection of retrieval and generation improves question answering
One Size Does Not Fit All! Dynamic Retriever and Generator Selection for RAG
Information RetrievalArtificial Intelligence
Summary
Many systems that answer questions by finding information and then writing answers use the same approach for every question, even though some questions are easier or harder than others. The authors found that using a stronger search method usually helps more than just spending more effort on writing the answer, but both have limits and don’t always help for all questions. They created a system called DRAG that picks the best way to search and write answers depending on each question. This can make answering questions both faster and more accurate compared to using one fixed approach every time.
What this means in practice
- •For qa system engineers: Reduce the time to answer questions by dynamically choosing search and generation settings per query, improving efficiency while keeping accuracy.
- •For customer support platform developers: Improve automated response systems by adapting information retrieval and answer generation methods to suit question difficulty for faster, better replies.
Authors
Neeraj Anand, Payel Santra, Partha Basuchowdhuri, Debasis Ganguly, Sumit Bhatia
Abstract
Retrieval-Augmented Generation (RAG) systems typically employ fixed retriever and generator configurations across queries, despite substantial differences in query complexity and information needs, leading to inefficient allocation of computational resources. While retrieval and generation adaptivity have been studied independently, their joint effect on end-to-end RAG performance remains underexplored. We systematically analyze how retriever and generator complexity interacts across factoid and multi-hop question answering (QA), including bridge and composition reasoning tasks. Our analysis shows that stronger retrieval generally yields larger gains than increased generation effort, but both exhibit diminishing and non-monotonic returns, indicating that higher-complexity configurations are not uniformly better across queries. Motivated by these findings, we introduce DRAG, a query-adaptive framework for selecting retriever-generator configurations. We first propose DRAG$_\text{QPP}$, a training-free routing approach that uses Query Performance Prediction (QPP) signals to guide retriever selection and perplexity-based measures over retrieved context to guide generator selection. We further introduce DRAG$_\text{SFT}$, a supervised routing approach that fine-tunes an LLM to jointly predict retriever-generator configurations. Across three LLM families and four QA benchmarks, \qpprag~achieves performance comparable to strong static RAG baselines while substantially reducing inference latency, whereas DRAG$_\text{SFT}$ consistently improves effectiveness over static and training-free adaptive baselines. Overall, DRAG demonstrates that jointly adapting retrieval and generation achieves a more favorable effectiveness-efficiency trade-off than static RAG pipelines.