SwitchSplat enables flexible 3D views of changing objects in scenes

SwitchSplat: Learning Discrete Object States from Casually Captured Scenes

Computer Vision and Pattern RecognitionGraphics

Summary

When people capture 3D scenes with objects that can move or change, such as doors opening or switches flipping, existing methods struggle to show these changes realistically. The authors developed SwitchSplat, a new way to represent scenes that separates static parts from objects that can switch between two states. This lets users combine these states freely to see new views of the scene, even configurations not seen before. SwitchSplat learns this from regular videos without needing any special labels or annotations.

What this means in practice

  • For augmented reality developers: Render interactive 3D scenes with objects that can be toggled or moved in real time from any viewpoint.
  • For video game artists: Create flexible scene assets where doors, switches, or props can be combined in new configurations beyond recorded footage.

Authors

Yizheng Xie, Yalda Foroutan, George Shramko, Daniel Rebain, Shrisudhan Govindarajan, Andrea Tagliasacchi

Abstract

3D Gaussian Splatting has made high-fidelity 3D reconstruction fast and accessible, yet existing representations assume a static world. This assumption breaks down in casual captures, where a user naturally interacts with the environment -- opening doors, toggling switches, or moving objects -- introducing discrete state changes that no current framework can model. Time-conditioned dynamic methods such as 4DGS conflate when a state was observed with what state the scene is in, limiting them to replaying seen sequences rather than composing arbitrary object configurations. We present SwitchSplat, the first 3DGS representation to address this gap via a base+slot decomposition: a shared static base captures the invariant scene, while each slot independently holds the two binary states (on/off) of one interactable object. Because the assignment of Gaussians to slots is fully differentiable, the model discovers the geometry of each state directly from video, without manual annotation. At inference, the user can freely combine slot states and render the resulting scene from any viewpoint. We validate SwitchSplat on a new casual-capture dataset and show that it outperforms static and dynamic baselines in novel-view synthesis quality, while uniquely supporting rendering of state combinations never seen during capture. Project page: https://switchsplat.github.io