Framework models lLM-guided search as sequential decisions for better retrieval
A Sequential Decision Framework for Evaluating LLM-Guided Search
Information Retrieval
Summary
Searching for useful information can be tricky when a computer has to decide what to look at next on its own. The paper describes a way to think about search as a series of simple steps: either choosing what’s already found or asking new questions to look for more. This helps compare different ways of searching and figure out which works best for finding lots of relevant stuff efficiently. The authors show their ideas work well especially when using large language models to help search more deeply.
What this means in practice
- •For search engine developers: Design search systems that decide when to explore new queries or examine existing results to find more relevant documents efficiently.
- •For knowledge management teams: Improve automated document discovery by guiding iterative search with policies that balance exploration and exploitation for high recall.
Authors
Siqing Huo, Andrew Parry, Debasis Ganguly, Mark Smucker, Charles L. A. Clarke
Abstract
Recent systems applying retrieval-augmented generation (RAG) and "deep research" iteratively generate queries, retrieve documents, and examine their contents, using the resulting information to guide subsequent search steps. When this process is carried out autonomously rather than by a human searcher, the system must make explicit decisions about which search trajectories to explore and what objective the search is intended to optimize. We introduce a simple framework for reasoning about such search policies, providing a principled approach to evaluation. The framework models the search environment as a set of ranked lists and represents behavior using two primitive actions: select, which advances judgment through an existing ranking, and spawn, which generates a new ranking through query generation or reformulation. Each action produces a document to inspect, yields information that updates the search state, and contributes a reward for a task-specific objective. Within this abstraction, a variety of existing retrieval strategies can be expressed as policies operating over a common environment, while common retrieval objectives can be represented through reward and weighting functions over the resulting search trajectory, and the resulting formulation admits a natural reinforcement learning interpretation. We illustrate the approach using high-recall information retrieval (HRIR), where the goal is to identify as many relevant documents as possible with limited judging effort, showing how the framework provides a unified way to describe and reason about strategies for both classical retrieval tasks and emerging LLM-guided search systems. Through experiments, we demonstrate that both the underlying retrieval method and the "deep research" policy employed can materially affect performance.