Diffusion model speeds up realistic shadow rendering in neural volumes
DiffusionShadow: Diffusion-based Shadow Caching for Neural Volume Rendering
GraphicsMachine Learning
Summary
Rendering shadows in 3D scenes that use neural networks typically takes a lot of computer power because calculating shadows requires many complex steps. The authors created a new method that compresses lots of shadow information into a single smart model that can quickly recreate shadows for many lighting angles without needing huge storage. This means shadows can be shown faster and still look very close to the high-quality versions. Their solution works well with existing 3D neural rendering systems and saves both time and memory.
What this means in practice
- •For graphics engineers: Render high-quality volumetric shadows in real time for medical imaging or scientific visualization using compact neural representations.
- •For game developers: Integrate fast, realistic shadows into volumetric effects in games without large memory costs from storing many shadows.
Authors
Kai-Chen Tung, Qi Wu, David Bauer, Mengjiao Han, Silvio Rizzi, Kwan-Liu Ma
Abstract
Implicit neural representations (INRs) have gained momentum in scientific visualization due to their compactness and scalability to large datasets, making them well suited for integration with direct volume rendering (DVR). However, real-time volume rendering of INR with advanced illumination effects, such as shadows, remains computationally expensive, as evaluating shadow terms via ray marching is costly. Alternatively, precomputing and storing shadows for many lighting directions is prohibitive in both memory and storage. To address this, we introduce a diffusion-based shadow caching framework that compresses a vast set of pre-calculated shadow INRs into a single diffusion model. Rather than focusing on generalizing to unseen directions, our method effectively memorizes and reconstructs a dense set of pre-trained lighting conditions on the fly. We first encode a collection of shadow coefficient volumes as shadow INRs, and then train a diffusion model conditioned on lighting direction to predict the corresponding shadow INR weights at inference time. This design integrates directly with standard INR renderers without additional runtime sampling. Experiments show that our approach achieves faster rendering than traditional methods while bypassing the massive storage bloat of independent INRs, producing shadows that closely match most of the reference results.