Multi-agent system generates realistic tumors with fine control on medical images
MATS: A Multi-Agent Framework for Controllable and Reflective Tumor Synthesis
Computer Vision and Pattern Recognition
Summary
Medical AI needs more data to recognize different tumors well, but real patient data is limited. The authors created a system called MATS that can generate realistic tumor images with detailed control over tumor shape and appearance. MATS uses a smart plan to create more examples of rare tumor types while keeping common ones frequent. It also checks its own work during generation to ensure the tumors match the desired features. This helps improve tumor detection AI even when trained on limited real data.
What this means in practice
- •For medical image analysis teams: Generate tailored synthetic tumor data to improve training of tumor segmentation models where real annotated data is scarce.
- •For medical imaging software developers: Create realistic tumor simulations for diagnostic tool testing and validation across different organ types.
Authors
Zhongnan Shi, Keyang Zhou, Siyi Li, Qichao Ying, Xinchen Ma, Shuyi Gong, Wei Tang, Sheng Li, Zhenxing Qian
Abstract
Synthesizing realistic tumors on medical images helps address data scarcity for clinical AI. Existing diffusion-based methods, however, lack fine-grained attribute-level controllability, closed-loop quality verification, and awareness of which morphological patterns are underrepresented in the target distribution. We present MATS, a multi-agent framework for text-controllable, cross-organ tumor synthesis. Given a target disease, MATS first profiles its morphological distribution over a fine-grained attribute space covering tumor geometry and appearance, and constructs a generation plan that amplifies underrepresented subtypes while keeping common ones dominant. It then localizes the target organ, proposes anatomically valid placement, and executes two-stage diffusion to produce the tumor mask and render texture, each conditioned on the corresponding attributes. A closed-loop reflective mechanism verifies attribute alignment at three checkpoints throughout synthesis and provides targeted feedback to guide re-generation when misalignment is detected. Trained exclusively on a single-source pancreatic neuroendocrine tumor dataset, MATS consistently improves downstream segmentation both in-domain and across LiTS, KiTS, and MSD-Pancreas benchmarks, whereas most prior generative augmentation methods fall below real-only training.