Dynamic 3d scenes reconstructed without unwanted occluders or artifacts

Reconstruct What You Can: Distractor-free 4D Gaussian Splatting

Computer Vision and Pattern Recognition

Summary

Capturing moving scenes on video can be tricky when things like shadows, smoke, or objects blocking the view get in the way. The authors present a new method called ReCan-GS that smartly separates real moving parts of a scene from these distracting elements. It uses a special score to detect uncertain areas and removes or separates them, allowing for cleaner dynamic 3D reconstructions. This approach works without needing extra manual masks or prior knowledge about the scene parts.

What this means in practice

  • For augmented reality developers: Create more stable and clearer dynamic 3D models from casual video streams by removing visual distractions like occluders and smoke automatically.
  • For visual effects teams: Improve the realism of dynamic scene reconstructions by filtering out transient occluders and artifacts without manual masking or pretrained segmenters.

Authors

Joel Julin, Liuyue Xie, Ananya Bal, Mose Sakashita, Koichiro Niinuma, Laszlo A. Jeni

Abstract

Dynamic reconstruction from casual video must preserve valid subject motion while suppressing transient occluders, shadows, and volumetric effects. Existing methods either suppress valid motion through static-scene assumptions or model all observations jointly, producing ghosting and unstable geometry. We introduce ReCan-GS, a 4D Gaussian Splatting framework that separates coherent scene content from inconsistent observations. It derives a Jacobian-based deformation-sensitivity score from the learned deformation field, using it as an empirical proxy for reconstruction ambiguity. The score guides separation into dynamic Core Gaussians and per-view Distractor Gaussians, or post-hoc pruning of sensitive Core Gaussians. The same criterion supports removal of spatial occluders, photometric artifacts, and evolving smoke or fog without masks, pretrained segmenters, or medium-specific supervision. Experiments on synthetic and real dynamic scenes and synthetic smoke/fog scenes demonstrate its usefulness across the evaluated distractor regimes.