Self-evolving vision language models improve physical reasoning accuracy
PhysEvolve: Verifier-grounded Self-evolution for Physical Reasoning with Executable Worlds
Artificial Intelligence
Summary
Physical reasoning helps machines understand how things move and interact, but current vision-language models struggle with answers about cause and effect in the physical world. The authors created PhysEvolve, a system where the model generates its own physics problems, checks them through simulations, and learns from the exact answers. This approach improves the model's ability to solve tricky questions about movement and changed scenarios, even for things it hasn’t seen before like cloth and fluids. It does all this without needing hand-labeled training data.
What this means in practice
- •For robotics engineers: Develop robotic systems with improved understanding of cause-effect in dynamic physical interactions for more reliable manipulation and navigation.
- •For simulation software developers: Create advanced physics-based training environments where models self-generate and verify complex tasks to improve physical reasoning capabilities.
Authors
Jinyuan Hu, Yilei Jiang, Chen Tang, Minghong Cai, Xinran Liu, Xiangyu Yue
Abstract
Physical reasoning underpins how embodied systems understand and act in the physical world, yet vision-language models (VLMs) remain unreliable on questions about dynamics and physical causality. Progress is critically constrained by the scarcity of grounded counterfactual training data, as observational videos capture only the factual trajectory that occurred. The self-evolving paradigm offers a route beyond fixed datasets by enabling models to construct their own training problems. Extending this paradigm to physics, however, requires an external source of truth that can both validate model-proposed environments and verify alternative physical outcomes under intervention. We introduce \physevolve{}, a verifier-grounded self-evolving framework that trains VLMs for causal physical reasoning from model-proposed executable worlds and simulation-authored tasks. At each round, a shared VLM proposes executable world programs and evidence schemas; a deterministic physics engine freezes each valid world, simulates matched baseline and body-elided counterfactual branches, and synthesizes discrete, answer-bound visual tasks. Exact verifier feedback from task solving updates both the shared policy via GRPO and an adaptive curriculum over physical intents. Starting from Qwen3-VL-8B-Instruct, \physevolve{} achieves targeted gains across PhysBench, CausalPhys, and ContPhy without human intervention data, improving PhysBench Throwing by $8.8$ points and ContPhy Counterfactual Removal and Goal-Pit by $7.4$ and $24.4$ points. These gains concentrate on interaction-sensitive and altered-world questions, demonstrating zero-shot transfer of causal principles to unseen complex dynamics (cloth, fluid, soft bodies) and real-world video.