FlexComposer: Unified Video Compositing from Images to Dynamic Footage with Flexible Trajectory Control
2026-07-31 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors developed FlexComposer, a new method to insert images or videos smoothly into existing videos while keeping motions and placements accurate. They created a way to separate an object's own movement from its overall position, making it easier to control where and how it appears. Their method also uses special tricks to place objects along chosen paths without extra complicated steps and learns lighting and shadows from a mix of simulated and real data. Tests show their approach is better at keeping motion realistic and matching user commands than previous methods.
Generative video compositingTrajectory-guided generationLatent spaceTranslation equivarianceVariational Autoencoder (VAE)Motion controlShadow harmonization3D reconstructionTemporal consistencyData curriculum learning
Authors
Songchun Zhang, Sitong Guo, Xianghao Kong, Pengwei Liu, Yuwei Guo, Lvmin Zhang, Anyi Rao
Abstract
Generative video compositing, which involves inserting external assets seamlessly into existing video sequences, is essential for content creation and visual effects. However, existing approaches suffer from a control-fidelity trade-off: they either hallucinate motion from static images, failing to preserve the dynamics of pre-animated assets, or lack fine-grained spatial control for precise asset placement along user-defined trajectories. We propose FlexComposer, a unified framework that standardizes video compositing as a trajectory-guided conditional generation task, enabling the seamless integration of both static images and dynamic footage. Our approach introduces three key designs: (1) a Unified Canonical Foreground Representation that decouples an object's intrinsic motion from its global displacement, standardizing heterogeneous inputs into a stabilized, centered latent space; (2) a Spatial-Aware Latent Injection strategy that exploits the translation equivariance of VAE latent spaces to transport canonical features onto target trajectories via a parameter-free mechanism; and (3) a Hybrid Dataset and Synthetic-to-Real Curriculum that synergizes procedural simulation, real-world cinematic footage, and generative data to implicitly learn physically plausible illumination and shadow harmonization. This unified design handles diverse inputs from product photos to dynamic subjects achieving high-fidelity motion control and environmental integration without the need for explicit 3D reconstruction or auxiliary learnable adapters. Extensive experiments demonstrate that FlexComposer outperforms state-of-the-art methods in visual quality, temporal consistency, and trajectory adherence.