Physically accurate video repair improves realistic video generation
Agentic Physics-Grounded Video Generation via Memory-Augmented In-Context Reasoning
Computer Vision and Pattern Recognition
Summary
Making videos that show realistic physical actions is hard because mistakes can only appear after the video is made. The authors created a method called PhysRepair that finds where the physical errors start and fixes only those parts without redoing the whole video. It learns from past fixes to improve over time and makes videos that follow the rules of physics better than before. This helps in creating videos that look and behave more realistically in different areas like fluids, heat, and light.
What this means in practice
- •For visual effects teams: Fix physical inconsistencies in generated video content to reduce reshoots and manual edits in production pipelines.
- •For simulation software developers: Integrate adaptive repair mechanisms to improve physical realism in physics-based video simulations across diverse domains.
Authors
Xikang Guan, Shaocong Yang, Yanqi Bao, Can Wang, Zheng Gu, Zhengqing Chen, Kai Chen, Jing Huo, Yang Gao
Abstract
Physical plausibility is essential for realistic video generation. Existing methods impose physical guidance before synthesis via simulations or refined prompts, but this can limit dynamic diversity, miss errors that emerge only after generation, and require repeated regeneration when inconsistencies remain. We therefore study a generate-diagnose-repair paradigm that preserves the original generation capability and selectively repairs physical inconsistencies afterward, raising two key questions: what to repair and how to repair? To address them, we introduce PhysRepair, an agentic video generation framework that combines causal reasoning over the observed physical failure with prior repair experience to enable localized and adaptive correction. Specifically, Spatiotemporal Causal Constraints identify the first invalid physical state and its causally affected states, producing a repair specification that guides physically plausible local regeneration while preserving unaffected content. Experience-augmented In-context Learning combines the ongoing repair context with relevant experience from previous repairs, allowing the agent to adapt its repair decisions across diverse physical failures and accumulate completed trajectories for future cases. Experiments on Physics-IQ Verified across solid mechanics, fluid dynamics, optics, thermodynamics, and magnetism demonstrate that PhysRepair consistently improves the physical plausibility of generated videos over existing methods. Our project page is available at https://xxkkxxx.github.io/PhysRepair-Website/.