Self-knowledge guides language models to reason more efficiently
SKIP: a Self-knowledge-guided Step-wise Preference Learning Framework for Concise Reasoning
Artificial Intelligence
Summary
Large language models often think in many steps to solve problems, but this can waste time and sometimes make answers worse. The authors designed a way to help these models decide when they have enough information to give a good answer earlier. Their method teaches the model to check its reasoning step-by-step and learn from which steps lead to better answers. This helps the model give shorter, faster explanations without losing accuracy. Their approach also works well on new kinds of problems it hasn't seen before.
What this means in practice
- •For ai developers: Reduce inference time and computational cost in large language model reasoning by enabling earlier confident answers.
- •For software engineers: Improve interactive AI systems by integrating a method that balances reasoning length and answer correctness dynamically.
Authors
Qinhong Lin, Yuhao Zhang, Yinglun Feng, Zhongliang Yang, Linna Zhou
Abstract
While Chain-of-Thought (CoT) reasoning has been proven to be effective, it often leads to overthinking, resulting in computational overhead, inference latency, and even degraded performance in large language models (LLMs). Existing concise reasoning frameworks significantly compromise accuracy while compressing the length of output. In this paper, we propose SKIP, a self-knowledge-guided step-wise preference learning framework. Starting with lightweight fine-tuning to adjust the model's output style, SKIP introduces a carefully designed knowledge probing mechanism to guide model to output an answer at each reasoning step. Based on the correctness of intermediate steps, we construct preference data that guide the model toward more efficient and correct reasoning by leveraging DPO. Experimental results demonstrate that our method effectively improves reasoning compression while mitigating performance degradation after fine-tuning. Besides, SKIP shows strong generalization ability on out-of-distribution datasets. We further conducted ablation studies on the component parameters of our framework.