Water motion captured in 3D for realistic looping views
Eulerian Motion Reconstruction for Water Scenery
Computer Vision and Pattern Recognition
Summary
This work addresses how to create realistic and looping animations of water scenes from just one short video. The authors developed a technique that reconstructs the movement of water in three dimensions and over time, so you can view it from different angles and it can loop smoothly. They use a method that simulates motion fields in 3D space plus adjustments for random waves and ripples. Their results show clearer and more believable water animations than previous methods that worked mostly in 2D.
What this means in practice
- •For computer graphics developers: Create photorealistic, loopable 3D water animations from simple 2D video inputs for interactive rendering in games and simulations.
- •For film visual effects teams: Generate realistic and view-dependent water motions from captured footage without needing complex manual modeling or exhaustive video sets.
Authors
Chuhan Chen, Yen-Chi Cheng, Ayush Saraf, Rajvi Shah, Tuotuo Li, Johannes Kopf, Chen Gao, Hung-Yu Tseng, Deva Ramanan, Matthew O'Toole, Changil Kim
Abstract
Reconstructing and animating water scenery from nature produces compelling and immersive visual experiences. Previous work examined this task from the perspective of 2D video textures, with the goal of creating a looping video. In our work, we tackle the problem from a 3D perspective, creating a looping 4D dynamic reconstruction which can be interactively rendered from novel viewpoints from a single non-looping 2D source video. We represent motion as a 3D static \textit{Eulerian} motion field that advects canonical Gaussian splats that are cyclically reborn at fixed time periods, supervised using rendering losses. To model non-periodic and stochastic dynamics present in real-world scenes, we add a non-periodic, time-varying residual term to capture deviations from the static Eulerian motion field. We show quantitatively and qualitatively that our framework enables photorealistic animation of water scenes better than prior art.