Conversational search improves by rearranging queries with fewer LLM calls
CompCQR: Compositional Query Generation for Training-Free Conversational Search
Computation and LanguageArtificial Intelligence
Summary
When people ask questions during conversations with AI, their questions can be unclear and depend on previous chat history. The authors show that reorganizing parts of these questions can help search engines find better answers without needing to call large language models many times. They created a method to build many queries from small pieces, improving search accuracy across multiple tests. Their method works with different AI and search systems and uses fewer resources while performing better.
What this means in practice
- •For search engine developers: Improve search result quality for multi-turn user queries by reorganizing query content efficiently without frequent large language model use.
- •For customer support platform teams: Enhance automated answer retrieval in chat-based support systems by generating better standalone queries from conversational context.
Authors
Yunah Jang, Kang-il Lee, Joongbo Shin, Kyomin Jung
Abstract
Multi-turn interactions with LLMs are becoming increasingly common in information-seeking scenarios. However, user queries are often ambiguous and context-dependent, making them ill-suited for direct use as retriever queries. Conversational query reformulation (CQR) addresses this issue by rewriting the current utterance into a stand-alone query grounded in the dialogue history. Recent LLM-based CQR approaches achieve strong performance; however, their repeated LLM invocations and misalignment with downstream retrievers remain challenges. In this work, we begin from the observation that retrievers are highly sensitive to content ordering: simply reordering the same content can lead to changes in retrieval coverage and performance. Based on this, we propose a novel training-free method that generates a very large number of queries with minimal LLM usage by compositionally combining a small set of atomic components. We further apply LLM reasoning to construct a high-quality document set that balances precision and recall while capturing the user's core intent. Our framework generalizes across both open- and closed-source LLMs as well as dense and sparse retrievers. It achieves strong performance on four widely used conversational benchmarks, with up to 22.5% relative MRR improvement over the previous state-of-the-art baseline with far fewer LLM calls.