Stereogaussians create 3d scenes from stereo photos without extra tuning
StereoGaussians: Feed-Forward 3D Gaussian Splatting from Stereo Images
Computer Vision and Pattern Recognition
Summary
Showing people new views of a scene from just two stereo photos is tricky because some parts of the scene appear for the first time and need to be imagined. The authors designed StereoGaussians to make a kind of 3D model directly from a stereo image pair using pretrained stereo networks to understand depth and appearance. Their method also predicts how to fill in areas not visible in the original pair with two layers of 3D Gaussian points. They trained the system on a large dataset and tested it on new scenes, showing it improves on existing methods for imagining new nearby views.
What this means in practice
- •For augmented reality developers: Create 3D scene representations from stereo camera pairs to render nearby views without per-scene optimization.
- •For robotics perception teams: Generate metric 3D models on the fly from stereo images to help robots understand their surroundings beyond their camera views.
Authors
Boyuan Tian, Huangying Zhan, Zhan Li, Shin-Fang Chng, Hanwen Yang, Zirui Wang, Yi Xu
Abstract
Feed-forward 3D Gaussian Splatting (3DGS) enables reconstruction without per- scene optimisation, but practical stereo-camera applications require nearby-view extrapolation beyond the input views. Stereo depth anchors visible surfaces, yet rendering newly exposed regions also requires learned appearance and additional scene capacity. We introduce StereoGaussians, which predicts a metric 3DGS representation from a single calibrated stereo pair. It reuses intermediate repre- sentations from frozen pretrained stereo networks to predict Gaussian attributes, while calibrated disparity anchors the geometry. A second Gaussian layer and an expanded image canvas provide capacity for disoccluded and outside-field-of- view content. For training, we construct SceneSplat-Stereo from quality-filtered 3DGS teachers, pairing stereo inputs with nearby target views across 803 training scenes. Experiments on unseen real and photorealistic stereo benchmarks demon- strate improvements over strong view-synthesis baselines, while ablation studies support our main design choices.