Align reasoning language with query language to improve monolingual question answering
Investigating the Role of Reasoning-Language Alignment in Monolingual Retrieval-Augmented Generation
Computation and Language
Summary
The paper looks at how using the same language for both thinking and answering questions helps language models do better. The authors studied this in a German-only system that answers niche questions by searching a large German text collection. They found that forcing the model to think in German improved answers compared to thinking in other languages, even if it was better at those other languages overall. However, thinking in German still didn’t beat the model’s natural ability to think in English, showing that truly strong multilingual reasoning needs to be native.
What this means in practice
- •For multilingual chatbot developers: Improve chatbot accuracy by matching the reasoning language to the language of user input and retrieved documents in retrieval-augmented systems.
- •For customer support teams: Enhance automated responses in non-English languages by aligning internal reasoning processes with the support language to better use retrieved information.
Authors
Oliver Hauck, Mario Sanz-Guerrero, Katharina von der Wense
Abstract
Reasoning traces improve large language models (LLMs), but current models are trained to reason mostly in English. It has been shown that forcing a model to reason in another language degrades accuracy, even when the reasoning language matches the language of the prompt -- but only for a setting where the model reasons over a short prompt. Here, we ask whether the same holds for retrieval-augmented generation (RAG), where the model must read and integrate a large amount of retrieved evidence in the target language. To study this, we build a fully monolingual German RAG question-answering testbed over the fictional world of the tabletop role-playing game The Dark Eye, a domain that is richly documented in German but too niche for the model to answer from memory, so that it has to rely on retrieval. Varying the forced reasoning language of an agentic RAG system on this testbed, we find that aligning the reasoning language with the language of the query and the retrieved documents helps. Forced German reasoning outperforms forced French, although the model benchmarks higher in French, so the benefit comes from alignment and not from language proficiency. The advantage grows when the retrieved context is richer and structure-aware. However, forced German only reaches the level of the model's native, unconstrained English reasoning without surpassing it, showing that native multilingual reasoning is needed. We publicly release the testbed and QA benchmark.