Physloop improves video generation with physics aware self correction
PhysLoop: Harnessing LLMs' Physics Awareness for Self-Improving Video Generation Models
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Video generation models create realistic videos but often fail to correctly show how things move and interact physically. This paper presents PhysLoop, a system that uses the knowledge inside large language models to help video models make more physically accurate videos. PhysLoop helps identify mistakes in generated videos and fixes them, allowing the video models to improve by themselves with less human help. The authors also provide a large set of videos and tools to evaluate how well their system works.
What this means in practice
- •For visual effects teams: Use PhysLoop to produce videos with more realistic physical motions and interactions, reducing manual fix work during production.
- •For augmented reality developers: Automatically improve physical consistency in generated scene videos to increase realism in AR experiences without frequent human corrections.
Authors
Yuxuan Chen, Xiaojie Li, Hao Chen, Juncheng Ma, Kewei Zhang, Zhongjing Du, Guoqiang Wei, Fangfang Wang, Zhen Dong, Zhiyang Dou, Liming Zhao, Yan Zeng, Jianwei Zhang, Daquan Zhou
Abstract
Video generation models produce visually realistic videos but still struggle with physical dynamics and causal consistency. Learning from generated videos requires fine-grained physical supervision, yet reliable quality judgments, correction pairs, and controlled variations are difficult to obtain. Large language models (LLMs) possess physical knowledge, but struggle to reliably guide video generation models in producing their training videos. We present PhysLoop, a unified harness designed to support physically grounded self-improvement in video generation models. The harness guides and constrains LLMs throughout video generation, selection, and correction, distilling their physical knowledge into physically consistent videos. The framework supports controllable training-data construction and inference-time video repair. By identifying and correcting failures, PhysLoop marks a key milestone toward recursive self-improvement (RSI) in video generation models with limited human intervention. We release 100K videos spanning diverse, realistic indoor and outdoor scenes, along with generation tools and two evaluation datasets: PhysLoop-Gen and PhysLoop-Repair. We evaluate controllable generation and localized repair on these datasets, respectively, and assess physical behavior on the external Physics-IQ Verified benchmark.