Binocular rendering sped up by reusing and patching a main image
From Mono to Stereo: Accelerating Binocular Gaussian Splatting via Reprojection and Selective Patching
Computer Vision and Pattern Recognition
Summary
Rendering two slightly different eye views of the same scene usually takes almost twice as long because each view is drawn separately. The authors propose a way to first fully render one eye’s view with special images called 2D Gaussian splats, then reuse and adjust that image to create the other eye’s view by filling in only missing parts. This approach saves time and memory while only slightly lowering the second eye’s image quality, which might be acceptable depending on how the images are used.
What this means in practice
- •For vr content developers: Create stereo 3D scenes efficiently by selectively re-rendering parts for the second eye to speed up VR rendering pipelines for static scenes.
- •For game engine programmers: Implement binocular rendering methods that reduce GPU memory and frame time by reusing a dominant-eye rendering with targeted patch updates.
Authors
Hongfei Zhu, Ling Zhou
Abstract
Binocular rendering requires two nearby views of the same scene and therefore repeats substantial visibility and shading work. We present a 2D Gaussian Splatting (2DGS) pipeline that fully renders a dominant-eye RGB image and an alpha-weighted depth proxy, reprojects that image to the affiliated eye, and repairs uncovered pixels. Small interior gaps are interpolated, whereas larger disoccluded regions are identified as regions of interest (ROIs) and selectively re-rendered. The depth proxy reuses the alpha-blending weights computed during dominant-eye rasterization, avoiding a separate depth-rendering pass. An adaptive ROI generator localizes the required updates using reprojected image boundaries and optional connected center-hole detection. On DTU, Tanks and Temples, and MipNeRF-360, the method reduces the measured time of a sequential two-pass binocular reference by 15.5\% to 28.8\% and peak GPU memory by 6\% to 11\%. The corresponding affiliated-eye quality degradation is at most 1.3 dB PSNR, 0.02 SSIM, and 0.02 LPIPS, representing a measurable trade-off that requires application-specific perceptual validation. These results establish a practical efficiency-quality trade-off for controlled static-scene stereo rendering and motivate future evaluation under continuous motion and on physical VR hardware.