RCEM: Embedder Equipped with Query Rewriting Skill for Robust Conversational Search in Distributional Shift
2026-06-01 • Computation and Language
Computation and Language
AI summaryⓘ
The authors introduce RCEM, a new conversational search model that helps AI understand multi-turn conversations better without needing to rewrite user queries each time. Instead of matching conversations directly to documents, RCEM aligns current queries with their rewritten versions to improve accuracy, especially when faced with new or different kinds of questions. Their method also avoids relying on hard-to-get training data about which conversations match which documents. Tests show that RCEM works better than existing methods, particularly when the conversations change in unexpected ways, and it can handle both regular and conversational searches using the same system.
conversational searchretrieval-augmented generationdense retrievalquery reformulationembedding modeldistributional shiftRecall@10QReCCTREC CAsTdocument retrieval
Authors
Kilho Son, Paul Hsu, Cha Zhang, Dinei Florencio
Abstract
Conversational search has become increasingly important in retrieval-augmented generation (RAG) systems, where users interact with AI assistants through multi-turn conversations containing context-dependent queries. We propose RCEM, a conversational dense retrieval model that distills the query reformulation capability of LLMs into the embedding model, enabling context-aware retrieval without explicit query rewriting during inference. Unlike prior conversational dense retrieval approaches that learn direct conversation-to-document matching, RCEM aligns conversational-query embeddings with rewritten-query embeddings, improving robustness under distributional shift. RCEM does not require conversational query-to-document relevance mappings for training, which are often expensive and difficult to obtain with high quality. Extensive experiments on QReCC, TopiOCQA, and TREC CAsT demonstrate that RCEM consistently outperforms strong conversational retrieval baselines, achieving particularly large gains under distributional shift, including up to 20% improvement in Recall@10. RCEM further extends the base embedding model with conversational query rewriting capability while preserving its original retrieval functionality, allowing both standalone and conversational queries to be encoded by a single model and searched against existing document indexes without rebuilding the retrieval database.