PhysTrace detects and explains physical errors in generated videos
PhysTrace: Interpretable Temporal Evaluation of Physical Consistency in Video Generative Models
Artificial IntelligenceRobotics
Summary
Video models try to predict what will happen next based on physical rules, but sometimes they make mistakes. The authors present PhysTrace, a tool that identifies when and where these physical mistakes happen in videos by comparing what the model expects with what actually appears. It can explain in simple terms what physical rules were broken, helping improve video prediction models. They tested PhysTrace on a new dataset that includes exact times when errors occur, showing it works better than earlier methods.
What this means in practice
- •For video model developers: Identify and describe exactly when video models break physical rules to improve their training and reliability.
- •For robotics engineering teams: Locate inconsistencies in video-based physical predictions used for robot planning to enhance action reliability.
Authors
Taekoan Yoo, Wonkyung Jung, Kyunghun Kim, Kyeongbo Kong
Abstract
World action models increasingly rely on pretrained video models assumed to capture physical dynamics for prediction, planning, and action generation. Existing evaluations assess plausibility, probe representations, or measure predictive surprise, but rarely identify which history-conditioned physical expectations conflict with observations. We introduce PhysTrace, a training-free framework aligning MLLM-generated, history-conditioned expectations with observed video windows to temporally localize physical violations and provide interpretable descriptions without task-specific probes or classifiers. We also introduce IntPhys2-Timing, adding violation intervals to IntPhys2. PhysTrace achieves a Hit Rate of up to 70.7% across physical principles on IntPhys2-Timing and outperforms the compared methods in overall possible-impossible discrimination accuracy on IntPhys2. Applied to reconstructed videos along the trajectories of pretrained diffusion/flow-based video models, PhysTrace reveals attribute- and noise-dependent patterns in physical violation localization, enabling interpretable physical consistency analysis across timesteps.