MorphoOrgaAgent automates organoid analysis with natural language input
MorphoOrgaAgent: A Foundation-Model-Based Multi-Agent System for Autonomous Organoid Analysis
Multiagent SystemsComputer Vision and Pattern Recognition
Summary
Studying tiny 3D cell clusters called organoids is important for understanding diseases and testing drugs, but analyzing their shapes usually takes a lot of manual work and coding skills. The authors created MorphoOrgaAgent, a computer system that understands what measurements and images a user wants from simple language, then automatically segments organoids from microscope pictures and generates detailed reports. This system works without needing prior training (zero-shot) and matches expert measurements closely. The authors also provide a benchmark to test such systems, and they share their code for others to use.
What this means in practice
- •For laboratory technicians: Automatically generate detailed organoid shape measurements and reports from microscopy images using simple language commands, reducing manual work and coding needs.
- •For biotech software developers: Integrate a multi-agent framework for zero-shot segmentation and analysis of organoid images into drug discovery platforms to speed morphological studies.
Authors
Hanyi Zhang, Maximilian Hoermann, Lion J. Gleiter, Yiling Xu, Bettina Katalin Budai, Hans-Ulrich Kauczor, Carsten Marr, Tingying Peng
Abstract
Organoids are three-dimensional tissue models whose morphology provides important insights into tumor development, disease progression, and drug testing. Extracting these morphological features relies heavily on manual segmentation, which is time-consuming and labor-intensive. Furthermore, performing quantitative statistical analysis typically requires custom coding skills and a mathematical background, presenting a major barrier for experimental biologists. To address these challenges, we introduce MorphoOrgaAgent, a multi-agent framework that achieves zero-shot organoid segmentation, automated data analysis, and report generation based on natural language input. The framework consists mainly of three core components: a TaskUnderstandingAgent that identifies requested measurements and visualization types; a hybrid segmentation module that combines Cellpose-derived geometric prompts with text prompts to guide SAM3 for zero-shot organoid instance segmentation; and a ReportAgent that computes quantitative metrics and compiles them alongside generated visualizations into a structured report. We further introduce MorphoOrgaVQA, a benchmark designed for quantitative evaluation of agent systems in organoid morphology analysis. Experimental results demonstrate that MorphoOrgaAgent handles both explicit and descriptive user requests, produces measurements closely matching ground truth, and generates complete analysis reports without requiring manual programming. The complete source code and MorphoOrgaVQA benchmark are publicly available at https://github.com/peng-lab/MorphoOrgaAgent.