Phydict reconstructs 3d lung ct scans from few x-rays using physics and ai
PhyDiCT: Plug-and-Play CT Reconstruction from Sparse X-Rays via Differentiable Rendering and Strong Priors
Computer Vision and Pattern Recognition
Summary
Reconstructing full 3D CT images from a few X-ray pictures is very hard because important 3D details are missing. The authors developed PhyDiCT, which combines a physics-based model of X-ray imaging with a powerful AI that knows about lung images to fill in missing parts without extra training. It creates 3D images that better match real X-rays and look more realistic by balancing physics and learned knowledge. Tests show it outperforms other methods, improving the quality of lung scans reconstructed from limited data.
What this means in practice
- •For medical imaging engineers: Reconstruct higher-quality 3D lung CT images from sparse X-ray views for improved diagnostics in limited-data scenarios.
- •For industrial nondestructive testing teams: Generate detailed 3D internal images of objects using fewer X-ray projections by combining physics and pretrained generative models.
Authors
Weicheng Dai, Shantanu Ghosh, Kayhan Batmanghelich
Abstract
Reconstructing 3D Computed Tomography (CT) images from a few X-ray projections is a highly ill-posed inverse problem due to the loss of volumetric information. We propose PhyDiCT, a training-free framework that integrates a differentiable Physics-based forward model, grounded in the Beer-Lambert law, with a text-conditioned Diffusion as a strong prior to reconstruct 3D lung CT images. We refer to our approach as training-free since the prior model is used without fine-tuning, and our goal is to steer the denoising procedure to generate samples consistent with X-ray observations. We guide the diffusion generation using Split Gibbs sampling to jointly optimize for projection fidelity (reward) and consistency with prior knowledge. Also, we introduce a test-time refinement step that enhances image realism and anatomical coherence. We extensively evaluate our method on publicly available 3D CT datasets using both perceptual and semantic metrics, demonstrating that it surpasses existing plug-and-play diffusion and fully trained reconstruction approaches. Our findings highlight that combining a strong generative prior with the underlying physics of image formation substantially improves reconstruction quality, e.g., 7.5\% improvement on SSIM compared to full training methods. Code will be released at https://github.com/batmanlab/PhyDiCT.