Study treats AI agents like people to improve their safety and reliability
Do AI Agents need Preventive Care? An Epidemiologic Cohort Design for Agentic AI
Artificial IntelligenceMultiagent Systems
Summary
AI agents are computer programs that work on their own to do tasks. This paper suggests studying these agents like doctors study groups of people over time to understand their behavior and outcomes. The researchers propose treating each agent’s work history and behaviors as if they were a person's health and lifestyle, to find what causes problems or makes them more reliable. They show this idea by looking at how curiosity or careful checking affects AI agent safety and accuracy. This method could help design better and safer AI systems by watching their actions continuously.
What this means in practice
- •For healthcare ai engineers: Track AI agent behaviors over time to improve detection of safety risks during clinical tasks.
- •For data validation teams: Use cohort design to identify behavioral patterns linked to data checking effectiveness in analytics pipelines.
A position paper. It proposes an approach and reports no results.
Authors
Shu Jiang, Graham A. Colditz
Abstract
As AI agents increasingly perform tasks autonomously across sectors, including healthcare and biomedical research, understanding how their behaviors develop and influence outcomes over time becomes increasingly important. Existing evaluations largely focus on model capabilities and performance on predefined tasks. Here, we propose an epidemiologic cohort design for studying agentic AI, drawing on principles established in human population studies. The framework treats individual agents as members of a cohort, their underlying model as genetic makeup, accumulated work as their time scale, and naturally occurring behaviors as exposures that may influence subsequent outcomes. By following agents longitudinally, this approach enables investigation of behavioral determinants of adverse events, estimation of their potential preventable burden under causal assumptions, and characterization of behavioral changes over an agent's work life. We illustrate the design through two prospective observational cohorts examining curiosity and probing as a safety question, and data checking and analytical accuracy as a reliability question. Together, the framework and illustrative studies provide a generalizable blueprint for designing epidemiologic cohorts of AI agents.