ScaleVid: Geometry-Aware Video Object Scaling with Mesh-Free Inference
2026-08-12 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed a new two-step method for resizing objects in videos that keeps the shape looking natural and the background unchanged. Their approach first learns to separate the moving object from the background, then applies 3D-like changes to the object without needing detailed 3D models. They use made-up examples created from real videos to train their system, avoiding the need for exact real-world before-and-after video pairs. Tests show their method works well and is faster than older techniques that rely on heavy 3D calculations.
video object scalinggeometry-aware transformationforeground-background composition3D deformationvideo synthesisanisotropic resizingmesh reconstructiondepth guidance
Authors
Youze Huang, Penghui Ruan, Bojia Zi, Xianbiao Qi, Shihao Zhao, Rong Xiao
Abstract
Geometry-aware video object scaling aims to anisotropically resize the object along object-centric axes while preserving geometric plausibility, temporal coherence, and background consistency. Existing text-guided methods mainly operate in the 2D image plane, while depth-guided approaches provide coarse control and mesh-based methods require costly 3D reconstruction. We present a progressive two-stage training framework that decouples geometry-aware foreground transformation from background preservation and realistic video composition, without mesh-pixel alignment and explicit 3D reconstruction at inference. In both stages, geometrically perturbed pseudo-sources are constructed from real videos, while the original complete videos are retained as reconstruction targets. The first stage uses planar transformations to learn robust foreground-background composition, whereas the second introduces object-centric 3D deformation guidance for geometry-aware scaling. This pseudo-source reconstruction formulation enables real-video synthesis without paired real-world scaling targets. We construct complementary paired-geometry and real-background benchmarks and further evaluate on in-the-wild videos. Extensive experiments demonstrate superior geometric consistency, foreground fidelity, and background preservation, together with faster and more practical inference than methods requiring explicit 3D reconstruction.