Large language models improve by training on questions that match their skill
STRETCH the Boundaries: A Unified Self-Taught Framework for Progressive LLM Evolution
Computation and Language
Summary
Large language models often get stuck improving because they keep practicing problems that are too easy or too hard. The authors propose a new training method called STRETCH that creates challenges just beyond the model's current abilities to keep it learning steadily. This method has two parts: one creates questions that push the model’s limits, and the other teaches the model to solve them better over time. Their experiments show this approach helps the model improve its reasoning skills more than traditional methods.
What this means in practice
- •For ai product developers: Train language model components with adaptive difficulty to enhance their problem-solving abilities in negotiation and operations research domains.$Commercial implications: Enables better performing AI agents for negotiation and optimization tasks, improving commercial software offering competitive advantages.
- •For machine learning engineers: Use dynamic challenge generation and co-training loops to stabilize and enhance training of large language models for reasoning tasks.
Authors
Yajie Yu, Mark Lee, Yue Feng
Abstract
Large language models (LLMs) often suffer from capability stagnation in self-improvement training because fixed difficulty levels fail to adapt to their evolving proficiency. To address this issue, we propose STRETCH (Self-Taught Reasoning Evolution via Targeted CHallenge), a unified framework inspired by cognitive scaffolding theory. STRETCH introduces a dynamic Stretch Zone mechanism that continuously aligns question difficulty with the model's solving capability. Within a single parameter space, the model alternates between a Scaffolder that generates adaptive, boundary-pushing challenges and a Learner that that optimizes its solving trajectories through reinforcement learning. This dual-loop co-evolution effectively stabilizes training, mitigates reward hacking and promote progressive reasoning growth. Experiments on both negotiation and operation research benchmarks demonstrate that STRETCH consistently outperforms strong prompting and domain-specific baselines. Further scaffolder configuration analysis shows that dynamic difficulty alignment is critical for sustained capability improvement and synchronized reasoning evolution.