ProCARE improves automation of real-world health studies with profile-based contracts
ProCARE: Real-World Study Automation via Profile-Grounded Evidence Contracts
Artificial Intelligence
Summary
Real-world studies use routine health data to understand treatments and risks outside of clinical trials. Automating these studies is tricky because health data can be messy and structured very differently across databases. The authors created ProCARE, a system that makes automation smarter by outlining each dataset’s details in an “Evidence Contract” and then converting study goals into clear, checkable steps. This helps avoid errors and keeps analyses consistent. ProCARE was tested on many tasks and worked better than typical language model approaches at producing correct and trustworthy study workflows.
What this means in practice
- •For health data teams: Automatically generate validated analyses of healthcare datasets with accurate clinical interpretations.
- •For pharmaceutical analytics teams: Streamline real-world treatment effect and safety studies by automating complex data mappings and checks across claims and EHR data.
Authors
Zhaoyu Fan, Bowen Han, Jincheng Ou, Yanwei Ren, Zehua He, Kaiyu Huang, Pengcheng He, Shanshan Ni, Chengchen Gong, Shuai Xue, Qingjiang Shi
Abstract
Real-world studies (RWS) analyze routinely collected healthcare data, such as electronic health records and insurance claims, to estimate treatment effects, monitor safety, and identify risk factors outside randomized trials. Automating RWS with large language model agents is appealing but brittle: valid analyses depend on dataset-specific clinical semantics, including index dates, follow-up windows, patient/event-level units, and outcome boundaries that are often implicit in observational data. This leads to schema mismatches, temporal errors, inconsistent analytical granularity, and unsupported clinical claims. We present ProCARE, a profile-grounded framework that reformulates RWS automation as constrained compilation. ProCARE converts heterogeneous data profiles into Evidence Contracts specifying available variables, temporal attributes, distributions, and evidentiary limits, and compiles study objectives into RWS-IR, a strongly typed intermediate representation for variables, time relations, analysis units, and statistical tasks. Deterministic validation and localized repair then check and revise generated workflows across schema mapping, cohort construction, variable derivation, analysis, and interpretation. On 36 EHR-based benchmark tasks, ProCARE outperforms strong LLM-agent baselines in code executability, semantic consistency, and conclusion traceability, with automated violation diagnoses showing moderate-to-substantial agreement with blinded expert review.