GraphicsComputer Vision and Pattern RecognitionDistributed, Parallel, and Cluster Computing
Summary
The authors created a way to render 3D images made of blobs (Gaussians) using a special chip called an Intelligence Processing Unit (IPU) that has many small processors with very fast but small memory. Each processor handles a part of the screen and communicates with neighbors to share data efficiently, following a strict schedule. They studied how limitations like memory size, communication speed, and uneven workload affect the speed and quality of the images. Their findings suggest new ways to design graphics processors that could work faster by better sharing data without relying on slower main memory. This work explores how future chips might handle 3D data more efficiently, especially in devices where saving memory and power is crucial.
Authors
Nicholas Fry, Ignacio Alzugaray, Mark Pupilli, Paul H. J. Kelly, Andrew J. Davison
Abstract
We present the first implementation of a 3D Gaussian renderer on an Intelligence Processing Unit (IPU), comprising 1,472 independent tiles with only on-chip SRAM; constraints that approximate properties of efficient sensor-processor architectures. Our input scenes are 3D Gaussian maps from real-world sequences. Each tile 'owns' a screen-space region of the framebuffer; Gaussian primitives are routed to destination tiles via Manhattan-distance hops on a north-east-west-south (NEWS) grid, then distributed to overlapping neighbours in an expanding tree pattern. Computation follows the IPU's Bulk Synchronous Parallel (BSP) model, with inter-tile communication defined at compile time. We show this hardware allows us to exploit spatial and temporal locality by enabling local data transfer between cores. We evaluate the bottlenecks in this SRAM-only implementation: inter-tile bandwidth, per-tile SRAM capacity, and workload imbalance from non-uniform Gaussian density. We analyse how these constraints affect performance and render quality. This exploration raises broader questions for conventional GPUs and 3D representations, suggesting that direct inter-SM (streaming multiprocessor) communication might offer ways to reduce DRAM access in GPU kernels. We discuss these implications for the future of on-sensor and DRAM-free architectures. Project page: https://nmjfry.github.io/ipu-3dgs/