Physics aware video generators improve fluid object interaction accuracy
Fluid-Gen-Zero: Grounding Pretrained Video Generators in Physics without Training
Computer Vision and Pattern RecognitionGraphics
Summary
Creating realistic videos of flowing fluids interacting with objects is hard, especially when you want the motion to follow the laws of physics. The authors show a new approach that uses a physics simulator to generate the movement while using existing pretrained video generators to make the images look real. This method doesn’t require new training and combines the best of both worlds for more accurate and believable fluid-object interaction videos. They also tested their method against others and found it consistently made the simulated motions closer to reality without losing video quality.
What this means in practice
- •For visual effects artists: Generate fluid-object interaction videos that accurately follow physics without costly retraining for realistic visual effects.
- •For robot motion planners: Simulate fluid and object interactions with higher physical fidelity to improve robot environment interaction planning.
Authors
Hong Huang, Yuqiu Liu, Chenyu You, Daniel Martin, Chuhang Zou, Wuyang Chen
Abstract
We present Fluid-Gen-Zero, a training-free framework for physics-aware fluid-object interaction video generation that decouples physical reasoning from appearance synthesis. Our key insight is to delegate motion dynamics to a physics simulator while preserving the appearance modeling capacity of pretrained video generators. We bridge these two domains through a two-level agentic workflow: generation-time planning, where a vision-language model (VLM) agent interprets intent and the simulation rollout to organize generation clips, and latent-space guidance, which injects simulation signals into denoising through region-aware latent wrapping. This plug-and-play design is compatible with current video foundation models. We further introduce a benchmark for fluid-object interaction video generation. Across Tora (CogVideoX-based), VACE and WanMove (Wan-based), Fluid-Gen-Zero consistently improves simulation alignment, reducing object trajectory error by 26.7%-81.5% and fluid fEPE (fluid flow endpoint error) by 67.9%-84.0%, while largely preserving perceptual quality. In a human preference study, raters favor Fluid-Gen-Zero in 55.1%-74.4% of same-backbone comparisons across three backbones, and in 90.4%-94.2% of comparisons against simulation-based methods. Code and data will be released upon acceptance.