Sparse view 3d gaussian splatting improved with few shot learning
Rethinking Sparse-View 3D Gaussian Splatting: A Few-Shot Learning Perspective on Overfitting Prevention
Computer Vision and Pattern Recognition
Summary
Creating 3D scenes from only a few pictures is hard because methods often get confused and produce poor results. The authors looked at this problem like a few-shot learning task, where each part of the 3D scene is treated like a tiny unit that can overfit the small data. They improved the process by creating better fake views to help training and by using special methods to stop the system from memorizing too much. Their approach won a top challenge by making sparse 3D reconstructions more accurate and stable.
What this means in practice
- •For augmented reality developers: Produce 3D scene reconstructions from few camera views that remain accurate and stable for AR experiences.
- •For telecommunication engineers: Improve real-time 3D scene modeling in video calls or live streams where camera viewpoints are limited.
Authors
Lejia Ye, Bin Wu, Jingjing Gao, Yiyang Wu, Xiaohu Zhang
Abstract
Sparse-view novel view synthesis (NVS) is essential for AR/VR and real-time communication applications, yet remains challenging when reconstructing 3D scenes from limited viewpoints. While 3D Gaussian Splatting (3DGS) excels in dense-view scenarios, it suffers from overfitting and geometry degradation under sparse inputs. Current approaches often tackle these problems from a local perspective, addressing either data quality or training regularization individually, lacking a unified and interpretable framework for systematic improvement. We introduce a new perspective that reframes sparse-view 3DGS from a few-shot learning viewpoint, treating each Gaussian primitive as a neuron and leveraging deep learning regularization techniques. Under this unified framework, our approach generates more effective supervision from scarce observations through two strategies: (1) data-related supervision enrichment via optical flow-guided pseudo-view synthesis and joint camera-scene optimization; and (2) training-time regularization via motion-aware dropout and early stopping, which suppress overfitting while preserving dynamic-scene stability. Extensive experiments on the SIGGRAPH Asia 2025 volumetric video challenge sparse-view dataset demonstrate significant improvements over state-of-the-art optimization-based and feed-forward baselines. Our system won first place in the sparse-view track of the SIGGRAPH Asia 2025 challenge.