Summary
Studying tiny 3D tissues called organoids helps scientists understand diseases and test medicines, but the process usually requires a lot of manual work and technical skills. The researchers created MorphoOrgaAgent, a computer program made of multiple smart helpers that can automatically identify and measure parts of organoids just by reading natural language instructions. It combines image recognition tools to spot organoids without needing training on specific examples, and it can produce detailed reports with numbers and pictures without any manual coding. They also built a test set called MorphoOrgaVQA to check how well such systems perform. This tool aims to make organoid analysis faster and easier for biologists who aren’t programmers.
organoidssegmentationmulti-agent systemzero-shot learningnatural language processingimage analysisCellposeSAM3quantitative metricsreport generation
Authors
Hanyi Zhang, Maximilian Hoermann, Lion J. Gleiter, Yiling Xu, Bettina Katalin Budai, Hans-Ulrich Kauczor, Carsten Marr, Tingying Peng
Abstract
Organoids are three-dimensional tissue models whose morphology provides important insights into tumor development, disease progression, and drug testing. Extracting these morphological features relies heavily on manual segmentation, which is time-consuming and labor-intensive. Furthermore, performing quantitative statistical analysis typically requires custom coding skills and a mathematical background, presenting a major barrier for experimental biologists. To address these challenges, we introduce MorphoOrgaAgent, a multi-agent framework that achieves zero-shot organoid segmentation, automated data analysis, and report generation based on natural language input. The framework consists mainly of three core components: a TaskUnderstandingAgent that identifies requested measurements and visualization types; a hybrid segmentation module that combines Cellpose-derived geometric prompts with text prompts to guide SAM3 for zero-shot organoid instance segmentation; and a ReportAgent that computes quantitative metrics and compiles them alongside generated visualizations into a structured report. We further introduce MorphoOrgaVQA, a benchmark designed for quantitative evaluation of agent systems in organoid morphology analysis. Experimental results demonstrate that MorphoOrgaAgent handles both explicit and descriptive user requests, produces measurements closely matching ground truth, and generates complete analysis reports without requiring manual programming. The complete source code and MorphoOrgaVQA benchmark are publicly available at https://github.com/peng-lab/MorphoOrgaAgent.