3D model generates controllable body and tumor CT segmentations
NV-Generate-Body: 3D Latent Diffusion Model forControllable Anatomy, Tumor, and Demographic Body Mask Synthesis
Computer Vision and Pattern Recognition
Summary
Creating large datasets of labeled CT scans is difficult and limited by privacy concerns. The authors developed a computer model that can generate 3D body maps showing organs and tumors in CT images, while controlling details like organ size, tumor presence, and patient traits such as sex and body mass index. This model uses a compressed representation to efficiently create high-resolution maps, even accurately representing tumor sizes despite limited tumor data. It was trained on thousands of samples and shows good alignment with real patient anatomy and tumor characteristics.
What this means in practice
- •For medical imaging developers: Create synthetic whole-body CT segmentation masks with controlled anatomy and tumors to augment data for training segmentation algorithms.
- •For radiology software teams: Generate diverse body CT label maps to improve robustness of analysis tools by simulating different organ sizes and tumor scenarios.
Authors
Can Zhao, Pengfei Guo, Dong Yang, Yufan He, Yucheng Tang, Ahmed Harouni, Benjamin Simon, Mason Belue, Stephanie Harmon, Baris Turkbey, Daguang Xu
Abstract
Curating large labeled CT datasets is expensive and constrained by patient privacy. We present NV-Generate-BodyMask, a 3D latent diffusion model that synthesizes whole-body CT segmentation masks with precise, joint control over organ anatomy, tumor burden, and patient demographics. The model operates in the compressed latent space of a frozen mask autoencoder, generating $256^3$ volumetric label maps conditioned on a 19-dimensional vector via Adaptive Group Normalization and cross-attention. A size-adaptive ROI loss with spatial halo weighting and a paired tumor injection pipeline enable accurate tumor size conditioning despite the scarcity of annotated tumor cases. Trained on 8,571 masks across 26 cohorts spanning 132 labels, the model achieves Spearman $ρ= 0.70$--$0.92$ for organ size, 85--100\% tumor recall for lung, pancreatic, colon, and bone lesions, and strong demographic alignment (BMI $ρ= 0.78$, sex-conditioned perfect prostate control).