Memory efficient training method cuts 3D scene rendering costs
BoundedGS: Memory-Efficient High-Resolution Training for 3D Gaussian Splatting
Computer Vision and Pattern Recognition
Summary
Training detailed 3D scenes usually needs a lot of computer memory, especially as the images get higher resolution. The authors propose a new way to organize memory use during training, which keeps the memory demand stable even as image resolution grows. This results in much less memory needed overall, faster training times, and similar quality in the final 3D scene. Their approach cleverly breaks tasks apart and stores data more compactly to avoid big memory spikes.
What this means in practice
- •For 3d graphics developers: Train high-resolution 3D Gaussian Splatting models with significantly less memory and time while maintaining image quality for realistic scene reconstruction.
- •For augmented reality engineers: Implement memory-bound high-resolution model training for efficient 3D scene representation in AR applications constrained by hardware limitations.
Authors
Xiaobin Deng, Changyu Diao, Min Li, Duanqing Xu
Abstract
High-resolution 3D Gaussian Splatting training increases pixel and Gaussian-tile intersection counts, expanding rasterization caches that coexist with projection and gradient state at peak memory. A complete-view update aggregates gradients across the view; each region's forward and backward caches can be released after its contribution is collected. BoundedGS separates rasterization and projection backward passes and bounds the active local rasterization working set according to its measured load. Candidate-indexed projection and compact gradient storage address the view-level peaks exposed by this change, while CPU-assisted structure rebuilding and bounded image supply manage resources outside rendering. In a resolution sweep at a fixed count of about 6M Gaussians, the measured rendering working set stays within 1.073-1.077 GiB from 1K to 16K; at 11K, a warm pass takes 19.79 seconds versus 21.76 for full-frame execution. Across five native 5K outdoor scenes, full training reduces the mean rendering working set from 9.97 to 1.76 GiB (82.3%), retains similar reconstruction quality, and shortens training time from 36.7 to 33.9 minutes. Project page: https://xiaobin2001.github.io/BoundedGS-Web/