Study treats AI agents like people to improve their safety and reliability

Do AI Agents need Preventive Care? An Epidemiologic Cohort Design for Agentic AI

Artificial IntelligenceMultiagent Systems

Summary

AI agents are computer programs that work on their own to do tasks. This paper suggests studying these agents like doctors study groups of people over time to understand their behavior and outcomes. The researchers propose treating each agent’s work history and behaviors as if they were a person's health and lifestyle, to find what causes problems or makes them more reliable. They show this idea by looking at how curiosity or careful checking affects AI agent safety and accuracy. This method could help design better and safer AI systems by watching their actions continuously.

What this means in practice

A position paper. It proposes an approach and reports no results.

Authors

Shu Jiang, Graham A. Colditz

Abstract

As AI agents increasingly perform tasks autonomously across sectors, including healthcare and biomedical research, understanding how their behaviors develop and influence outcomes over time becomes increasingly important. Existing evaluations largely focus on model capabilities and performance on predefined tasks. Here, we propose an epidemiologic cohort design for studying agentic AI, drawing on principles established in human population studies. The framework treats individual agents as members of a cohort, their underlying model as genetic makeup, accumulated work as their time scale, and naturally occurring behaviors as exposures that may influence subsequent outcomes. By following agents longitudinally, this approach enables investigation of behavioral determinants of adverse events, estimation of their potential preventable burden under causal assumptions, and characterization of behavioral changes over an agent's work life. We illustrate the design through two prospective observational cohorts examining curiosity and probing as a safety question, and data checking and analytical accuracy as a reliability question. Together, the framework and illustrative studies provide a generalizable blueprint for designing epidemiologic cohorts of AI agents.