Conversational search improves by rewriting queries in multiple steps
Learning Multi-Step Query Rewriting via Corpus Feedback for Conversational Search
Information Retrieval
Summary
Answering questions in a conversation can be hard because each question depends on what was said before. This paper shows a way to rewrite questions step-by-step, using information from the search results found earlier to improve later rewrites. The method learns to decide when to stop rewriting and turns conversational questions into clear standalone queries that work well with different search engines. The authors tested their approach on several datasets, and it worked better than other methods without needing human-written examples for training.
What this means in practice
- •For search engine engineers: Build conversational search systems that iteratively refine user queries using retrieved documents to improve answer accuracy without extra annotated data.
- •For virtual assistant developers: Enhance virtual assistants to better understand follow-up questions by rewriting queries in multiple steps grounded on search results for more accurate replies.
Authors
João Coelho, Hong Wang, Jie Yuan, Zhuoer Wang, Samson Koelle, Wei Niu
Abstract
Conversational Query Rewriting (CQR) turns a context dependent user turn into a standalone query for a retriever, and most methods do this in a single step from the dialogue history before retrieving once. The rewrite is therefore fixed before any corpus evidence is available to correct its reference resolution or its vocabulary. We recast CQR as a sequential retrieval problem: an agent rewrites the current turn, retrieves, and conditions its next rewrite on the returned passages. The agent acts in a typed space of three rewriting operations, resolving conversational intent into a standalone query, generating lexical reformulations, or synthesizing pseudo-documents for document-to-document matching, together with a stop action that ends the episode. We train the policy with supervised fine-tuning followed by reinforcement learning against a single retrieval-quality reward, using no human rewrite annotations. Across TopiOCQA and QReCC, the agent outperforms several retrieval-aligned baselines, while remaining effective across retrieval backends and generalizing to the CAsT benchmarks without additional training. Further analysis shows that, through retrieval-reward optimization alone, the learned policy develops a behavior of grounding pseudo-documents in passages retrieved by earlier steps, substantially improving retrieval.