Video diffusion enables more flexible consistent 3D scene editing
VidEdit3D: Flexible 3D Gaussian Editing with Video Diffusion
Computer Vision and Pattern Recognition
Summary
Editing 3D scenes seen from many angles is tricky because changes can look inconsistent from different views. The authors treat 3D editing like editing a video that shows moving viewpoints, allowing them to use powerful video-based AI models to keep changes smooth and consistent across angles. They introduce methods to choose key views, apply edits consistently without extra training, and limit changes only to the targeted parts of the scene. This approach results in clearer and more accurate 3D edits that better follow instructions.
What this means in practice
- •For 3d artists and animators: Create and modify 3D scenes with consistent edits from multiple viewing angles using video-based diffusion models.
- •For augmented reality developers: Apply controlled updates to 3D content in AR environments while maintaining realistic views from different directions.
Authors
Yanming Yang, Chenxi Song, Tong Zhao, Juntao Xu, Wenhao Rao, Yiwen Chen, Yiwei Wang, Chi Zhang
Abstract
Recent advancements in text-guided 3D editing, particularly with 3D Gaussian Splatting (3DGS), have shown promise but still lag behind the flexibility of 2D editing tools. Existing methods face a critical trade-off between preserving multi-view consistency and allowing structural modifications. Image-based editors can also struggle to apply the intended edit to objects viewed from unusual angles. To address these limitations, we introduce VidEdit3D, a framework that reformulates 3D scene editing as a trajectory-based video editing task. By drawing a parallel between camera movement in static 3D scenes and temporal dynamics in video, we leverage the robust spatiotemporal priors of pre-trained Video Diffusion Models (VDMs) to encourage semantic and structural coherence across views. Specifically, we propose Trajectory-Guided Rendering (TGR) to select informative keyframes and construct smooth video clips aligned with VDM input requirements. To propagate edits from an anchor view to these clips, we develop Flow-Aligned Video Editing (FAVE), a training-free framework incorporating an Editing Consistency Guidance (ECG) mechanism. ECG contrasts source and edited image conditions at a shared latent state, using scheduled source-image amplification to strengthen adherence to the edited reference while maintaining original scene context. Finally, we introduce Region-Selective 3DGS Refinement (RS-GS) to strictly confine geometric and appearance updates to target primitives, reducing background attribute leakage. Experiments demonstrate stronger instruction alignment and better multi-view consistency than the baselines.