Video diffusion models improve camera control with geometry aware test time scaling
GeoART: Geometry-Aware Refinement via Test-Time Scaling for Camera-Conditioned Video Diffusion
Computer Vision and Pattern Recognition
Summary
Camera-conditioned video generation often struggles to keep the camera angles accurate and consistent across different views because the model doesn't strongly enforce 3D geometry during the process. The authors introduce GeoART, a method that improves this by testing multiple options during generation and choosing the one that best fits geometric clues, without needing extra training or 3D data. This approach makes the video output look more realistic and consistent when viewed from different angles. GeoART works by refining decisions at certain steps using geometry checks and carefully picking the most reliable ones.
What this means in practice
- •For video production teams: Enhance camera angle control and consistency in AI-generated videos without retraining the model or requiring additional 3D data.
- •For augmented reality developers: Improve virtual camera alignment and geometry correctness in real-time AR video synthesis tasks using pretrained video diffusion models.
Authors
Liudi Yang, Ziyuan Liu
Abstract
Camera-conditioned video diffusion models often suffer from inaccurate camera control and geometric inconsisten- cies because camera constraints and estimated geometry are only weakly enforced during denoising. Existing reme- dies often require costly geometry-aware fine-tuning or ad- ditional 3D supervision. We introduce GeoART, a training- free self-refinement framework that improves camera con- trollability through geometry-aware test-time scaling. At selected denoising steps, GeoART generates multiple can- didates within the Predict-and-Perturb cycle and eval- uates them using a geometry prior, without directly modify- ing the denoising trajectory. A confidence gate suppresses unreliable geometric evidence, while rank-calibrated prob- abilistic selection combines score magnitude and relative rank to choose the next refinement state. Experiments demonstrate consistent improvements in camera controlla- bility, multi-view consistency, and visual quality over exist- ing sampling and self-refinement methods, without training or modifying the pretrained video diffusion model