Automated system improves clinical trial eligibility criteria using textual evidence
RECITE: Revising Eligibility Criteria Incorporating Textual Evidence
Artificial Intelligence
Summary
Many clinical trials fail because their rules for who can join are too strict. The authors created a system called RECITE that uses past trial changes and documents to suggest better, evidence-backed rule adjustments. They tested how well large language models could suggest improvements and found some models perform close to expert humans. This approach helps turn medical knowledge into practical trial design advice.
What this means in practice
- •For clinical trial designers: Automatically refine patient eligibility rules using past trial amendments to reduce enrollment failures and improve trial participation rates.
- •For medical knowledge engineers: Integrate clinical literature evidence into trial design recommendations by matching publications with trial criteria adjustments.
Authors
Russell Ro, Reza Abbasi-Asl
Abstract
Overly restrictive eligibility criteria contribute to widespread enrollment failure in clinical trials, yet published recommendations for how specific criteria should be broadened remain disconnected from trial design at scale. We introduce RECITE (Revising Eligibility Criteria Incorporating Textual Evidence), a framework for automated discovery, extraction, and implementation of evidence-based eligibility criteria revisions. RECITE uses protocol amendment documents from ClinicalTrials.gov to construct a dataset of 11,913 curated instances across 5,735 trials, each pairing original criteria with documentary evidence and target revisions. We benchmark fifteen LLM configurations under truncated-to-context-window and retrieval-augmented generation conditions, on a 3,116-instance sample under an LLM-as-judge protocol validated against ten domain experts (judge vs. human quadratic-weighted Cohen's $κ_{qw} = 0.694 \pm 0.096$). The best open-weight model, Gemma 2 9B, achieves $κ_{qw}$-aligned ordinal score $3.0 \pm 0.9$ on a 0--4 scale (normalized 0.742) and the best closed-weight model, GPT-4o, reaches $3.0 \pm 0.8$ (normalized 0.745). Using an agentic literature discovery system, we then identified 44 paper--trial matches across five publications providing trial-design and EC-modification evidence. Our findings suggest that existing clinical knowledge can be operationalized into actionable trial design recommendations.