Post-training reveals distinct improvements in large language model math reasoning
Beyond Compression: Diagnosing How Post-Training Changes Mathematical Reasoning
Machine LearningArtificial Intelligence
Summary
Mathematical reasoning in large language models improves after additional training, but measuring what exactly changes is tricky. The authors studied different training methods and found two patterns: for easier math problems, training mostly makes the model's answers easier to find; for harder problems, training actually improves the model’s top possible answers. They also found that training helps non-English math reasoning but does not close the language gap. They conclude that improvements from training are sometimes about efficiency, but not always.
What this means in practice
- •For machine learning engineers: Design evaluation protocols beyond simple accuracy to diagnose training effects on reasoning models across languages and problem difficulties.
- •For ai product teams: Improve math reasoning features in AI products by selecting training methods that expand solution coverage for hard problems rather than just speeding answer sampling.
Authors
Hongyang Li, Yiming Zhu, Xiao Li, Caesar Wu, Said Mammar, Pascal Bouvry
Abstract
Post-training is central to mathematical reasoning in modern large language models (LLMs), but endpoint pass@1 alone underidentifies what has changed. Gains may reflect newly reachable solutions, cheaper sampling of latent solutions, surface robustness, or memorisation. We compare three post-training paths under a common diagnostic readout: our sufficiently trained off-policy distillation trajectories, released Qwen3 off-policy-plus-on-policy distillation endpoints, and a released DeepSeek-Math endpoint trained with Group Relative Policy Optimisation (GRPO). Our probe uses cross-surface pass@K over verbatim prompts, paraphrases, numerical isomorphisms, and translations, plus consistency, distribution-shape, and verified supervised-fine-tuning (SFT) membership analyses. We find two regimes. On easier AMC problems, large-K ceilings are near saturation, so post-training mainly compresses sample cost. On harder AIME problems, post-training expands the large-K ceiling over the base model: sufficient off-policy distillation already raises this ceiling, Qwen3 released endpoints raise it further, and DeepSeek-Math GRPO does not dominate sufficient off-policy distillation at large K. English-dominant distillation improves non-English reasoning but preserves language-tier gaps. A controlled-overfit audit finds limited sensitivity in current SFT-membership probes. Compression is one regime of post-training, not a universal explanation.