Physics training data generation improves large language model accuracy
TIRE: Top-Down Formula Instantiation with Reliable Execution for Physics Problem Synthesis
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Physics problems for training AI often need expert work or can contain mistakes when created by AI alone. The researchers developed TIRE, a system that uses known physics formulas and checks calculations in advance to make accurate and varied physics problems. This approach helps train AI models better and makes them more reliable when solving physics questions. Their data and training method showed improved performance on several physics tests.
What this means in practice
- •For ai training data teams: Generate large sets of accurate, formula-based physics problems for training AI models with verified answers to improve reasoning abilities.
- •For educational software developers: Create physics learning tools with internally consistent problem sets backed by reliable numerical solutions to enhance teaching quality.
Authors
Xiaoyu Wu, Zhaoxin Yu, Xinglin Xiao, Nan Xu
Abstract
Advancing physics reasoning in large language models requires scalable training data that connect physical laws, concrete problem scenarios, and verifiable numerical answers. Expert-designed problems offer top-down control over physical laws, conditions, and scenarios, but require costly human effort. LLM-based synthesis from texts scales efficiently, but model-generated answers and assessments leave supervision vulnerable to hallucinations. We introduce TIRE, a framework for Top-Down Formula Instantiation with Reliable Execution that brings knowledge-driven problem construction to large-scale data generation. Top-down instantiation uses reusable physical formulas and their applicability conditions to control the physical relations and numerical configurations of generated problems. Reliable execution establishes reference answers before problem generation, providing a common numerical foundation for problem construction and teacher distillation. Together, these mechanisms combine control over problem content with execution-grounded supervision for both supervised fine-tuning and reinforcement learning. Using TIRE, we construct TIRE-19k, spanning nine subfields of physics. Across 3 backbone LLMs and 4 benchmarks, TIRE-19k outperforms 5 representative datasets with equal amounts of SFT data, exceeding the strongest baseline by 2.86 percentage points on average. Further GRPO raises Qwen3-8B's average accuracy by 1.52 percentage points over its TIRE-19k SFT checkpoint. These results demonstrate the value of generating physics training data from explicit physical knowledge with reliable execution.