ClinVerse enables realistic training of AI medical diagnosis agents
ClinVerse: A Realistic Multi-Agent Diagnostic Environment for Training and Evaluating Medical Agents
Artificial IntelligenceMultiagent Systems
Summary
Diagnosing diseases using AI is hard because most training uses simple question-answer sets that don’t capture the back-and-forth of real doctor-patient interactions. The authors created ClinVerse, a system that simulates real medical scenarios using six different clinical roles interacting together, based on a large database of real patient cases. This allows AI agents to learn how to ask the right questions, order exams, and make better diagnoses over time. They showed that various AI models improve their diagnostic skills when trained and tested within ClinVerse.
What this means in practice
- •For healthcare ai developers: Train AI agents that can interact dynamically with patients and clinical staff to improve medical diagnosis accuracy.
- •For clinical decision support teams: Evaluate and benchmark diagnostic AI models in realistic multi-role clinical scenarios before deployment.
Authors
Sirry Chen, Fangke Chen, Xuanwen Ding, Wei Chen, Zhongyu Wei
Abstract
Disease diagnosis is a key application of large language model (LLM) agents in healthcare. However, existing medical agents are primarily trained and evaluated with static question-answering data, providing limited opportunities to learn active information acquisition, examination decision-making, and dynamic clinical interaction. To address this limitation, we present ClinVerse, an interactive, real-data-grounded multi-agent environment designed for training and evaluating medical diagnostic agents. Built on ClinPatient, a scalable dataset comprising 150k diagnostic cases derived from real-world electronic health records, ClinVerse supports coordinated interactions among six specialized clinical roles. Specifically, a doctor agent starts from an initial patient complaint, progressively gathers clinical evidence through patient interviews and examination requests, and ultimately produces a diagnosis. For each interaction trajectory, a rule-based scoring framework quantifies information acquisition, examination ordering, and diagnostic accuracy. These trajectory-level scores serve as reward signals for agent training while also enabling scalable evaluation. In experiments, we evaluate various general-purpose and medical-domain models and show that nine self-evolution methods across three categories consistently improve agents' diagnostic capabilities within ClinVerse. We further analyze the interplay among model capability, environment feedback, and self-evolution methods.