ByteSplat speeds up 3D scene training by cutting GPU data movement
ByteSplat: Efficient Distributed 3D Gaussian Splatting Training via Intra- and Inter-GPU communication reduction
Distributed, Parallel, and Cluster Computing
Summary
Training detailed 3D scenes takes a lot of computer memory and time, especially when using multiple graphics cards (GPUs). The authors found that moving data between memory and GPUs, and between the GPUs themselves, slows down the process. They created ByteSplat, a system that reduces this data movement by combining steps, pruning data intelligently, and sending only important information between GPUs. This approach makes training faster while keeping the quality of 3D scenes high.
What this means in practice
- •For 3d graphics developers: Train large-scale photorealistic 3D scene reconstruction models faster by reducing the data movement bottlenecks across multiple GPUs.
- •For machine learning infrastructure teams: Optimize multi-GPU training workloads that handle dense geometric data by cutting unnecessary memory access and communication overhead.
Authors
Shuo Wu, He Zhu, Han Zhao, Xiaohui Zhang, Yaqian Zhao, Hui Wei, Ruyang Li, Hongzhi Shi, Lihua Lu, Jingwen Leng, Yu Feng, Minyi Guo
Abstract
3D Gaussian Splatting (3DGS) enables photorealistic scene reconstruction, but training large-scale scenes requires substantial memory and computation. Distributing training across multiple GPUs increases available memory capacity, yet its performance is strictly constrained by data movement. We identify two dominant bottlenecks: repeated off-chip memory accesses during forward and backward rasterization, and inter-GPU communication of partial Gaussian gradients. To address these bottlenecks, we present ByteSplat, a distributed 3DGS training framework that jointly reduces intra- and inter-GPU data movement. First, ByteSplat fuses forward rasterization and backward rasterization into a single GPU kernel, retaining intermediate results on-chip to eliminate redundant off-chip transfers. Second, to alleviate the increased on-chip storage by fused rasterization, ByteSplat introduces hardware-aware pruning that considers both rendering quality and per-tile shared-memory constraints, increasing the overall performance of fused execution. Lastly, ByteSplat exploits gradient sparsity to eliminate zero partial gradients from inter-GPU communication. GPU-efficient encoder and decoder kernels compact the remaining records and directly aggregate the received gradients on their owner GPUs, reducing both communication volume and local aggregation overhead. We evaluate ByteSplat across six datasets. Compared with the baseline, ByteSplat reduces intra-GPU off-chip traffic and backward inter-GPU communication volume by 63.4% and 65.8%, respectively. On eight GPUs, ByteSplat achieves up to 6.1$\times$ training speedup while preserving reconstruction quality.