Supervised fine tuning data curated 100 times faster for math and physics
From Weeks to Hours: Fast and Principled SFT Data Generation for Reasoning LLMs
Machine Learning
Summary
Large language models can solve complex problems by thinking through multiple steps, but teaching them this skill takes a lot of carefully chosen teaching examples, which can be expensive and slow to prepare. The authors designed a new method called SFTProbe that quickly predicts the quality of these teaching examples without generating and checking huge amounts of data. Their approach cuts down the time for preparing training data from weeks to just hours, while still improving or matching previous results in math and physics problem-solving. They also created a new physics reasoning dataset that led to a model beating a top competition silver medalist.
What this means in practice
- •For machine learning engineers: Cut training data preparation time from weeks to hours by evaluating data quality efficiently for reasoning tasks.
- •For automated tutoring developers: Create models that solve complex math and physics problems more accurately by using smaller but carefully selected fine-tuning datasets.$Commercial implications: Enables building high-quality automated tutoring systems that outperform existing models in reasoning accuracy.
Authors
Hongyi Henry Jin, Wenhan Yang, Meysam Ghaffari, Carlos Morato, Baharan Mirzasoleiman
Abstract
Reasoning-oriented post-training enables large language models (LLMs) to solve complex tasks via multi-step inference. Supervised fine-tuning (SFT) on high-quality reasoning traces is a particularly efficient approach, but its effectiveness depends critically on careful data curation. Existing pipelines rely on a costly generate-then-filter paradigm: large pools of long reasoning traces are produced by multiple teacher models and subsequently filtered for difficulty and diversity using additional LLMs, often requiring weeks of computation. In this work, we propose SFTProbe, a fundamentally different approach that bypasses this pipeline by directly predicting the quality of reasoning data. We show that a brief LoRA-based adaptation, combined with evaluating loss on only 5% of each reasoning trace, suffices to estimate both difficulty and diversity. SFTProbe is grounded in a geometric view of post-training, where pretrained LLMs lie in a low-rank, anisotropic loss basin. By probing this basin via low-rank perturbations, we construct compact loss signatures whose clustering captures diversity, while loss at curvature inflection points provides a robust proxy for difficulty. We also provide a simple criterion for teacher selection. Empirically, our approach reduces data curation time from weeks to hours while maintaining or improving performance. Remarkably, on mathematical reasoning, 5k generated examples match the highly filtered 89k-example in OpenThoughts-Math dataset, and 7.5k generated examples outperforms it by 2.5% in accuracy. We further introduce a high-quality physics reasoning dataset, and use it to train a model that outperforms an IPhO 2025 silver-medal-winning model. The code and the dataset are released at https://bigml-cs-ucla.github.io/SFTProbe-page/.