Human study programs run using configurable ai participants

Executable Human Study Programs with Silicon Participants

Human-Computer Interaction

Summary

It can be hard to repeat or build on human studies because their procedures are written as text instead of code. The authors created a framework that turns a study’s description into a computer program that can run with AI-based participants simulating humans. This helps researchers check study methods, test ideas before real experiments, and analyze behaviors of AI in simulated studies. Social scientists tested the tool and found it useful for understanding and prototyping studies.

What this means in practice

  • For social science research teams: Run and test human study protocols using AI participants to prototype and understand experiments before involving real human subjects.
  • For ai behavior analysts: Evaluate how AI participants respond in simulated human studies to investigate AI decision patterns and interaction behaviors.

Authors

Xuan Liu, Haoyang Shang, Yuanjun Feng, Shao-Yu Chu, Qiyu Li, Yash Raj Shrestha, Haojian Jin

Abstract

We introduce HumanStudyProgram, a framework that represents a documented human study as an executable program with configurable LLM-based silicon participants. Its computational abstraction separates study logic from participant implementation through standardized APIs for presenting information and collecting responses. We implement a researcher-facing authoring tool that reconstructs source-linked study descriptions from papers and supplementary materials, supports researcher review, and generates executable HumanStudyProgram artifacts. Researchers can then configure silicon participants, run the programs, and analyze results from aggregate findings to individual responses. We demonstrate the approach through study cases and evaluate the tool through a hands-on expert review with 12 social science researchers. All completed the workflow and identified potential uses in understanding published protocols, prototyping studies, and examining AI behavior. We aim to improve the reproducibility of human studies by reconstructing their documented procedures as programs that others can reuse and build upon.