Diverse Thinking Schemata Elicit Better Reasoning in Large Language Models
2026-06-08 • Artificial Intelligence
Artificial Intelligence
AI summaryⓘ
The authors study how large reasoning models solve math problems by looking at their 'thinking patterns,' which include the steps they take and the different answer paths they try. They find that models that explore more varied thinking patterns perform better. To improve this, they create a method called DiScO that teaches the model to recognize and increase these diverse thinking patterns using reinforcement learning. Their tests show that DiScO helps the model solve problems more accurately and fix mistakes better. This work highlights how encouraging variety in thought processes can boost reasoning ability.
Large Reasoning ModelsMathematical ReasoningReasoning TransitionsAnswer CandidatesThinking SchemataReinforcement LearningPolicy OptimizationDiversity in AIInferenceError Recovery
Authors
Xinyue Liang, Yizhe Yang, Yu Bai, Bin Xu, Jiawei Li, Yang Gao
Abstract
Large reasoning models (LRMs) have attracted increasing attention for their ability to solve complex mathematical problems by generating extended reasoning chains. In this work, we focus on two critical yet underexplored aspects of the reasoning process: reasoning transitions capturing the distinct transitions between reasoning steps and answer candidates reflecting the variety of solution paths produced by the model. We collectively define these two aspects as thinking schemata. We observe a correlation between the diversity of thinking schemata and model performance, which motivates us to enhance diversity as a means to further improve reasoning potential. To this end, we propose Diverse Schemata Policy Optimization (DiScO), a framework that first endows the model with schemata awareness, then encourages diversity through reinforcement learning, and further promotes diverse reasoning at inference time. Experiments on multiple mathematical reasoning benchmarks demonstrate that DiScO consistently outperforms standard group relative policy optimization. Beyond accuracy, human-annotated analyses show that DiScO substantially improves the model's ability to recover from erroneous initial attempts. Overall, our work suggests the important role that diversity of the thinking schemata plays and points to scaling along the diversity dimension as a promising research direction.