PaMoSplat: Part-Aware Motion-Guided Gaussian Splatting for Dynamic Scene Reconstruction
2026-05-11 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionGraphicsRobotics
AI summaryⓘ
The authors present PaMoSplat, a new method to better reconstruct and track moving 3D scenes by breaking the scene into parts and using motion information from videos. They start by grouping parts from multiple camera views and then use motion data to track how these parts move over time. Their approach improves how well the 3D scene looks and how accurately parts are tracked, while also being faster to train. They tested it on various real-world scenes and found it works better than previous methods and enables editing the scenes in 4D.
dynamic scene reconstructionGaussian splattingoptical flow3D segmentationmulti-view clusteringmotion priorsrigid motion estimationdifferential evolutionary algorithmrendering loss4D scene editing
Authors
Yinan Deng, Jianyu Dou, Jiahui Wang, Jingyu Zhao, Yi Yang, Yufeng Yue
Abstract
Dynamic scene reconstruction represents a fundamental yet demanding challenge in computer vision and robotics. While recent progress in 3DGS-based methods has advanced dynamic scene modeling, obtaining high-fidelity rendering and accurate tracking in scenarios with substantial, intricate motions remains significantly challenging. To address these challenges, we propose PaMoSplat, a novel dynamic Gaussian splatting framework incorporating part awareness and motion priors. Our approach is grounded in two key observations: 1) Parts serve as primitives for scene deformation, and 2) Motion cues from optical flow can effectively guide part motion. Specifically, PaMoSplat initializes by lifting multi-view segmentation masks into 3D space via graph clustering, establishing coherent Gaussian parts. For subsequent timestamps, we leverage a differential evolutionary algorithm to estimate the rigid motion of these parts using multi-view optical flow cues, providing a robust warm-start for further optimization. Additionally, PaMoSplat introduces an adaptive iteration count mechanism, internal learnable rigidity, and flow-supervised rendering loss to accelerate and optimize the training process. Comprehensive evaluations across diverse scenes, including real-world environments, demonstrate that PaMoSplat delivers superior rendering quality, improved tracking precision, and faster convergence compared to existing methods. Furthermore, it enables multiple part-level downstream applications, such as 4D scene editing.