Cost of Reasoning in non-English Languages: A Case Study on Japanese
2026-07-11 • Computation and Language
Computation and LanguageArtificial IntelligenceMachine Learning
AI summaryⓘ
The authors explored whether a language model can be trained to reason well in Japanese instead of English, which usually has more reasoning training data. They created a Japanese version of a large language model and tested it on tasks like coding, math, and science. Their results showed that it is possible for the model to reason in Japanese, but its performance is generally only as good as English-based models. Interestingly, the model did not do better on tasks related to Japanese culture, meaning reasoning in Japanese alone doesn't improve cultural understanding automatically.
Reasoning Language ModelsLanguage Model PretrainingJapanese LanguageQwen-3-8BGRPOMultilingual ModelsModel EvaluationCultural Benchmarks
Authors
Yuu Jinnai
Abstract
Reasoning Language Models (RLMs) achieve their strongest performance when they reason in English, the language for which reasoning-oriented training data is most abundant. However, reasoning trace is a clue for model interpretability and safety, and useful in practice for both the model users and for model developers. Thus, it is desirable to be able to develop a model that reasons in a language of the user's choice, while still maintaining strong reasoning performance. To this end, we study the feasibility of training a model that reasons in Japanese. We develop a Japanese-reasoning variant of Qwen-3-Swallow-8B, which is a Japanese LLM continually pretrained from Qwen-3-8B, with GRPO and evaluate it across coding, math, and science benchmarks. The study shows that reasoning-language control is feasible by training a Japanese continually pretrained model with GRPO. However, its performance is at best on par with strong English-reasoning baselines on several benchmarks. We also evaluate the trained model on Japanese cultural benchmarks and observe that the model's performance is worse than the baseline models, suggesting that the reasoning in Japanese does not immediately improve performance on culturally relevant tasks for free.