Photorealistic ct volume renderings converted into 3d gaussian splats without training
CTBakeGS: training-free baking of photorealistic CT volume renderings into 3D Gaussian splats
Computer Vision and Pattern RecognitionGraphics
Summary
Photorealistic images of CT scans help doctors understand anatomy, but showing these images usually needs powerful computers and special software. The authors developed a way to turn CT images directly into 3D models made of colored dots called Gaussian splats without needing long training times. Their method runs fast enough to show these detailed 3D images in real time even on regular laptops. This makes it easier to interactively explore medical images with good detail and speed.
What this means in practice
- •For medical imaging professionals: Create interactive 3D visualizations of CT scans quickly on standard computers without needing extensive training times or powerful graphics cards.
- •For medical software developers: Integrate fast conversion of CT volume data into real-time 3D Gaussian splat models for applications like surgical planning or educational tools.
Authors
Haruki Fukuda, Constantin Lux, Wolf Schweitzer, Tadashi Hosoya, Lars Ebert
Abstract
Photorealistic volume rendering of computed tomography (CT) may convey anatomy intuitively, but it needs the CT data, dedicated software and a powerful graphics processor. Three-dimensional Gaussian splatting (3DGS) displays photorealistic scenes in real time on less powerful devices, but existing medical approaches train each scene for minutes to hours. We aimed to convert a CT volume rendering directly into a 3DGS model without training. Splats were placed on an adaptive octree of millimetre-sized cells. Their colour and opacity were computed on the graphics processor with the renderer's lighting, cell opacity was converted analytically into splat opacity, and faint splats were retained stochastically above the 1/255 opacity floor of standard 3DGS rasterisers. Six public CT scans (1.5-mm voxels) were converted with three transfer functions and compared with converged references by peak signal-to-noise ratio (PSNR), structural similarity and a learned perceptual metric. Two-million-splat models reached a PSNR of 32.6 +/- 0.9 dB for bone, 30.1 +/- 1.0 dB for contrast-enhanced vessels and 32.3 +/- 1.5 dB for skin and bone. A conversion pass (usually two or three per model) took 4-30 s on a current laptop and 9-60 s on a desktop with a 4-GB graphics card. On the laptop, the models rendered at 68 frames per second at 800 x 800 pixels. Training-free conversion produced real-time 3DGS models of photorealistic CT renderings in about a minute or less per conversion pass on ordinary computers, with fidelity that we consider sufficient for interactive viewing.