FactorSplat enables flexible color control in medical volume images

FactorSplat: Appearance-Controllable Gaussian Proxies for Medical Volume Rendering

Computer Vision and Pattern Recognition

Summary

Medical volume images show detailed views of the inside of the body, but it can be hard to change colors and visibility of different areas quickly. The authors developed FactorSplat, a method that lets you control colors and transparency on the fly for specific regions in CT and MRI scans. This method keeps the shape and shading consistent while allowing different color settings without retraining the model. It delivers better image quality and fast rendering, making it easier to explore medical scans with different appearances.

What this means in practice

  • For medical imaging developers: Implement interactive volume rendering tools that allow clinicians to change color and opacity settings quickly without retraining for each visualization preset.
  • For graphics engine developers: Integrate fast, flexible Gaussian-based volume rendering that supports real-time editing of appearance parameters in large 3D medical datasets.

Authors

Zhongpai Gao, Benjamin Planche, Meng Zheng, Anwesa Choudhuri, Terrence Chen, Ziyan Wu

Abstract

Transfer functions (TFs) control color and visibility in medical volume rendering, but image-trained Gaussian proxies typically bake one transfer function into their appearance. We present FactorSplat, a per-scene N-dimensional Gaussian splatting (N-DGS) proxy that accepts region-specific intensity-to-RGBA curves at inference. A local lookup applies the authored color and opacity change, while a shared functional encoder and low-rank per-Gaussian factors learn the residual appearance response. Geometry and directional appearance remain shared across presets, with visibility control and TF-aware pruning preserving the ability to hide and reveal structures. On seven CT and MR scans, FactorSplat improves mean PSNR and changed-region error over region-aware VEG across validation, interpolation, unseen composition, and out-of-distribution (OOD) edits. Across these four splits, seven-scan mean PSNR gains over VEG range from 1.10 to 1.52 dB. One checkpoint per scan supports unseen edits without retraining. At $1600^2$, the cached fast renderer averages 524 FPS with 1.17 ms TF switches. Project page: https://gaozhongpai.github.io/FactorSplat/.