Large language models learn where to redirect reasoning for self-correction
Inject a Turn, Not a Token: Learning Where to Redirect LLM Reasoning for Self-Correction
Computation and LanguageArtificial Intelligence
Summary
Large language models sometimes make mistakes when thinking through problems. Instead of changing the model itself, the authors teach a small part to figure out where the model should be guided to rethink its answers. They found it works better to give a new instruction as a fresh question turn, which helps the model change its reasoning path. This method, called UPTURN, improves accuracy on math problems without retraining the main model.
What this means in practice
- •For chatbot developers: Redirect conversations internally to fix model reasoning errors without retraining the whole system.
- •For automated math tutoring platforms: Improve step-by-step solution accuracy by smartly injecting corrective prompts when the model produces mistakes.
Authors
Jiwon Jeon, Myungsik Cho, Jeonghye Kim, Youngchul Sung
Abstract
Large language models struggle to reliably identify and correct errors in their own reasoning. Unlike prior training-based approaches that fine-tune an LLM to generate revisions or feedback, we learn where to intervene in failed rollouts and determine how to redirect them. To choose where to intervene, we train a 130K-parameter MLP head on a frozen encoder using only final-answer correctness. For redirection, we find that under a fixed generation budget, the tested inline injections fail to reliably redirect reasoning, whereas presenting the same instruction in a new user turn changes the model's course. The resulting correction framework, UPTURN (User-Prompt TURN injection at a learned position), requires no LLM fine-tuning: it retains a selected prefix, injects an instruction as a new user turn, and regenerates the suffix. Across four reasoners and six math benchmarks, UPTURN improves average accuracy in every setting over a 32-rollout baseline. It outperforms self-correction and reflection-token injection baselines under matched correction budgets, transfers across reasoners and to GPQA-Diamond without retraining, and complements GRPO and self-correction methods.