Large language models improve problem solving with backward reasoning
Construting Reverse Thinking: Developing Large Language Models' Reverse Thingking Ability
Artificial Intelligence
Summary
Solving hard problems normally means thinking step-by-step forward, but this can miss important details or mistakes. The authors found common reasons why AI models make errors on math problems. They trained models to also think backward from the solution, like checking steps in reverse, by giving examples and rewards for both forward and backward thinking. This helped models get better at solving complex math challenges more flexibly and accurately.
What this means in practice
- •For math problem software developers: Build AI tools that solve complex math problems more accurately using backward reasoning training.
- •For ai product teams: Create AI assistants with improved reasoning flexibility by incorporating forward and backward thinking modes.$Commercial implications: Enables development of advanced AI reasoning features for commercial math tutoring and problem-solving apps.
Authors
Xin Liu, Yunhai Li, Chunfu Jia, Ziliang Chen, Jisen Song
Abstract
When facing complex problems, humans tend to try various ideas for different issues. Human thinking patterns exhibit remarkable flexibility in adapting to diverse scenarios. GPT-o1, GPT-o3, and DeepSeek-R1 adopt long chain-of-thought models to address complex problems by increasing reasoning depth, which default to a forward reasoning mode. We conducted statistical analysis on the accuracy of different mathematical problem datasets on models of different scales, and found five reasons for errors: Insufficient solution-space coverage, Computational mistakes, Unverified assumptions, Ignoring constraint conditions, Maximum response length limitation. To address the above issues, we proposed a backward reasoning pattern construction method aimed at enhancing the model's reverse thinking ability and dynamic adaptability. First, we constructed an easy-hard two-stage Math dataset for training large models and gradually improving their inference ability at different difficulty levels. The dataset contains forward reasoning paths as well as backward reasoning paths. And a two-stage supervised fine-tuning process is applied to progressively train the model's backward reasoning capability. Furthermore, a fine-grained reward mechanism is developed, employing smoothed reward signals to strengthen the model's ability to autonomously select thinking modes during the reasoning process, thereby avoiding reward hacking. A linear-decay balanced sampling strategy is designed to maintain a balance between forward and backward reasoning path samples during training, enabling the model to converge quickly and stably. Experimental results show that our method significantly improves reasoning efficiency and accuracy in tasks such as mathematical proofs, offering a flexible and efficient reasoning paradigm for solving complex problems.