Constant-memory methods improve differentiable light tracing simulations
Constant-Memory Differentiable Light Tracing
Graphics
Summary
Simulating how light behaves in 3D scenes is important but can use a lot of memory, especially when calculating how changes affect the lighting (differentiation). The authors show that existing memory-saving methods for tracing light from the viewpoint don't work well when tracing from the light source because light paths can affect many pixels. They propose two new techniques that keep memory use low while still accurately calculating gradients needed for tasks like adjusting lights or materials. These methods help efficiently simulate complex light effects, potentially speeding up graphics tasks like inverse rendering.
What this means in practice
- •For graphics engineers: Improve memory efficiency when computing gradients for light simulations involving complex caustics using light tracing approaches.
- •For visual effects studios: Enable faster inverse rendering workflows by using low-memory differentiable light tracing to recover scene properties from images.$Commercial implications: Supports production tools that infer lighting and materials automatically, reducing manual adjustment time and cost.
Authors
Linas Beresna, Eugene Fiume
Abstract
Reverse-mode differentiation of Monte Carlo light transport naively requires storing a computation graph whose size grows with path length, making it impractical for deep or high-sample simulations. Path Replay Backpropagation (PRB) eliminates this memory complexity for viewpoint path tracing by reconstructing paths from their random seeds and performing a constant-memory backward replay. However, this formulation relies on a one-path-one-pixel property that does not hold for light tracing: a single light path can contribute to many pixels, and the corresponding per-vertex adjoints cannot be recovered during replay from constant-size state. We demonstrate that this structural difference prevents a direct extension of PRB to light tracing in constant memory, and that buffered two-pass variants retain linear memory scaling with path length. We then introduce two constant-memory reverse-mode formulations for differentiable light tracing. The first, Reservoir Light Replay Backpropagation (ResLRB), compresses the set of per-path sensor connections into a single stochastically selected representative using weighted reservoir sampling, preserving a two-pass structure at the cost of increased gradient variance. The second, LRB-3-pass, introduces an additional traversal that accumulates downstream adjoint contributions prior to backpropagation, retaining all connections and matching naive reverse-mode AD in expectation. Both methods support detached and attached formulations, including differentiation of sensor splat positions induced by geometric perturbations. We validate correctness against naive AD, characterize memory and runtime trade-offs, and demonstrate inverse rendering of specular caustics driven entirely by light tracing.