ABCD: Alpha-Composited Block Coordinate Descent: Constant-VRAM Training for Large Radiance Fields
Computer Vision and Pattern RecognitionGraphics
Summary
The authors introduce ABCD, a new way to train 3D scenes made of alpha-composited parts, here applied to 3D Gaussian Splatting. Their method breaks the scene into blocks and trains one block at a time while freezing others, using pre-rendered images to represent inactive parts. This reduces the video memory needed so it doesn't grow with scene size, allowing larger scenes to be trained on smaller GPUs. Their tests show this approach keeps nearly the same image quality as traditional methods with only a small drop in accuracy.
Authors
Ka Heng Shiu, Kartic Subr
Abstract
We present ABCD (Alpha-Composited Block Coordinate Descent), an out-of-core training framework for alpha-composited radiance fields, instantiated here for 3D Gaussian Splatting. Our method reformulates training as block coordinate descent over spatial partitions: only one block of parameters is active at a time, while all others are frozen. By exploiting the associativity of alpha blending, these inactive regions can be pre-rendered and collapsed into foreground and background RGBA images. As a result, for fixed partition size and image resolution, peak VRAM becomes O(1) with respect to total scene extent, rather than growing with full scene size. This enables GPUs with limited memory to train scenes that would otherwise not fit in core. In experiments, our method closely preserves the reconstruction quality of 3DGS, with less than 5% PSNR degradation, while ABCD with compositing ablated suffers roughly 40% degradation. Our code can be found at https://github.com/shiukaheng/abcd