Long-context gaussian method improves 3d scene reconstruction from videos
LonGaussian: Tracking Foundation Priors for Long-Context Feedforward Gaussian Reconstruction
Computer Vision and Pattern Recognition
Summary
Reconstructing detailed 3D scenes from long videos is important for robots and AI systems. Existing methods struggle with processing many views at once due to big memory needs and repeated data. The authors present LonGaussian, which processes video in chunks and smartly merges repeated surface information, reducing memory and storage while improving quality. Their approach also filters out incorrect parts that block correct surfaces. Tests show LonGaussian creates better 3D views using fewer components than previous fast methods.
What this means in practice
- •For robotics engineers: Build real-time 3D maps from long video streams to improve robot perception and navigation with efficient memory use.
- •For augmented reality developers: Create compact, high-quality 3D scene models from video for AR experiences that require rendering novel views smoothly over extended sequences.
Authors
Zhoufeng Xie, Guanjun Wu, Shuai Zhang, Xinggang Wang, Wenyu Liu
Abstract
Reconstructing high-quality, renderable 3D scenes from long videos is essential for robotic perception and embodied intelligence. Many feed-forward Gaussian reconstruction methods rapidly predict 3D scenes from a small set of images, but extending them to longer sequences presents two bottlenecks: jointly processing all views increases GPU memory requirements, while accumulating pixel-aligned predictions redundantly represents the same surfaces and increases storage and rendering costs. We present LonGaussian, a streaming feed-forward framework that reconstructs compact, globally renderable Gaussian scenes from long image sequences with known intrinsics and unknown camera poses. Built on a pretrained streaming 3D reconstruction backbone, LonGaussian decodes long-context geometric features into dense Gaussians. Within each chunk, multi-frame point tracks link repeated observations of the same surface and guide a learned merger to consolidate the corresponding Gaussian predictions. Rendered-depth consistency filtering further suppresses Gaussians from other chunks that incorrectly occlude locally reconstructed surfaces. On DL3DV and RealEstate10K benchmarks, LonGaussian improves novel-view rendering quality while using substantially fewer Gaussians than the evaluated feed-forward baselines.