Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning
2026-08-05 • Computation and Language
Computation and LanguageMachine Learning
AI summaryⓘ
The authors study tasks that need a model to use different skills step-by-step, like math then planning. They create Skill Entropy, a way to measure how hard it is for a model to switch skills, and build Skill²-Bench, a large test set for such tasks. They find models struggle more as switching difficulty (skill entropy) increases. To improve this, they develop Skill-Entropy RL, a training method where the model learns to predict both answers and the skills used, boosting performance significantly. This approach also works on other math data, showing the idea is helpful beyond just their new benchmark.
long-horizon reasoningskill switchingSkill EntropySkill²-Benchreinforcement learningQwen modelsmulti-step taskscross-skill taskstraining signal
Authors
Yinghui He, Ling Yang, Jiarui Liu, Yongjin Yang, Lechen Zhang, Yingcheng Wu, Zhenfei Yin, Mengdi Wang, Sanjeev Arora
Abstract
Long-horizon reasoning in recent LLMs demands that the model switch between distinct skills inside a reasoning chain, such as first doing a math derivation, then using the result to plan a schedule. We call such problems cross-skill long-horizon tasks: multi-step tasks whose steps require different reasoning skills and depend on earlier outputs. Existing benchmarks often evaluate individual skills, lacking a principled way to measure how well a model switches between skills. We address this gap from both the evaluation and training sides. We introduce Skill Entropy, a measure of the difficulty of switching from one skill to another. We then propose Skill^2-Bench, a benchmark of cross-skill long-horizon tasks built over 558 skills across 9 verifiable and open-ended domains. Each task is assigned a task-level skill-entropy score and grouped into three difficulty levels. Evaluating 8 frontier and 4 open-source models on Skill^2-Bench reveals a skill-switching gap: accuracy decreases on higher-entropy tasks. We then turn skill entropy from a benchmark scale into a training signal. We propose Skill-Entropy RL, an RL framework where the model predicts not only the answer at each step but also the skill used to produce it. The reward combines step-level correctness with a skill-entropy reward that measures the alignment between the model-predicted skill sequence and the gold skill sequence. On Qwen3-4B-Instruct and Qwen3-1.7B, Skill-Entropy RL improves the Skill^2-Bench score from 34.4% to 68.4% and from 14.6% to 40.1%, respectively, outperforming competitive baselines. The same pipeline can be applied to off-the-shelf training data such as OpenR1-Math, indicating that skill entropy is a reusable training signal. Code available at: https://github.com/Gen-Verse/Skill-Entropy-RL