TRACE: High-Fidelity 3D Scene Editing via Tangible Reconstruction and Geometry-Aligned Contextual Video Masking
2026-04-01 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors introduce TRACE, a system that helps edit 3D scenes in videos more accurately by using detailed 3D shapes as guides. Their method allows users to make specific changes to parts of objects, like moving or swapping components, while keeping the overall shape intact. TRACE works in three parts: creating consistent 3D markers from different views, aligning new 3D objects precisely, and blending these changes smoothly into videos over time. Tests show that TRACE handles a wider range of edits and maintains structure better than older methods.
3D geometryvideo diffusionmesh editingmulti-view consistencyscene coherence3D-anchor synthesisrigid registrationvideo renderingautoregressive pipelinetemporal stability
Authors
Jiyuan Hu, Zechuan Zhang, Zongxin Yang, Yi Yang
Abstract
We present TRACE, a mesh-guided 3DGS editing framework that achieves automated, high-fidelity scene transformation. By anchoring video diffusion with explicit 3D geometry, TRACE uniquely enables fine-grained, part-level manipulatio--such as local pose shifting or component replacemen--while preserving the structural integrity of the central subject, a capability largely absent in existing editing methods. Our approach comprises three key stages: (1) Multi-view 3D-Anchor Synthesis, which leverages a sparse-view editor trained on our MV-TRACE datase--the first multi-view consistent dataset dedicated to scene-coherent object addition and modificatio--to generate spatially consistent 3D-anchors; (2) Tangible Geometry Anchoring (TGA), which ensures precise spatial synchronization between inserted meshes and the 3DGS scene via two-phase registration; and (3) Contextual Video Masking (CVM), which integrates 3D projections into an autoregressive video pipeline to achieve temporally stable, physically-grounded rendering. Extensive experiments demonstrate that TRACE consistently outperforms existing methods especially in editing versatility and structural integrity.