StreamSplat: Streaming Feed-Forward 3D Gaussian Splatting
2026-08-03 • Computer Vision and Pattern Recognition
Computer Vision and Pattern Recognition
AI summaryⓘ
The authors introduce StreamSplat, a method for creating 3D scenes from a sequence of images that arrive one by one. Unlike previous approaches that need all images upfront, StreamSplat updates the 3D scene step-by-step using a special memory system that grows only with the scene size, not the number of images. They add new techniques to guide the model with previous geometry and information for better predictions. Tests show StreamSplat works well even with fewer views and handles long sequences without running out of memory, improving as more images come in.
3D Gaussian SplattingNovel-view synthesisCausal processingVoxel-Aligned Causal CacheDepth anchoringFeature injectionCost-volume estimationFeed-forward networksScene reconstructionReal-time rendering
Authors
Changhao Song, Yuxuan Wang, Qibiao Li, Youcheng Cai, Ligang Liu
Abstract
Feed-forward 3D Gaussian Splatting enables efficient novel-view synthesis without per-scene optimization, but most existing methods assume a fixed set of context views and process them jointly. This limits their applicability to online scenarios where calibrated views arrive sequentially and the scene must be updated causally. We present \emph{StreamSplat}, a streaming feed-forward 3DGS framework that incrementally maintains a persistent geometry-grounded scene state and decodes it into renderable 3D Gaussians after each input chunk. StreamSplat centers on a \textbf{Voxel-Aligned Causal Cache (VACC)}, which stores historical 3D tokens in a memory-bounded voxel structure so that memory grows with explored scene geometry rather than stream length. To better reuse history during causal prediction, we introduce \textbf{History-Projected Depth Anchoring (HPDA)} to project cached geometry as depth guidance for current cost-volume estimation, and \textbf{Cache-Guided Feature Injection (CGFI)} to inject cached latent evidence into Gaussian-token regression. Experiments on DL3DV, RealEstate10K, and ScanNet show that StreamSplat remains competitive with state-of-the-art feed-forward 3DGS methods under sparse causal inputs, despite not using future views or full-scene context. More importantly, it scales to long input streams with 256, 512, and 1024 views where fixed-view baselines run out of memory, yielding sustained improvements in novel-view synthesis quality as more observations arrive. The code will be made publicly available upon acceptance.