Inspectable simulated interview participants help improve qualitative research protocols
Inspectable LLM-Simulated Participants for Qualitative Interview Rehearsal
Human-Computer Interaction
Summary
Preparing for interviews in social research can be tricky because it's hard to know if your questions are good before talking to real people. This paper shows how large language models (LLMs) can simulate interview participants to practice and refine interview questions. The researchers created a tool called Rehearsim that lets researchers interact with these simulated participants and review how well their interview plans work, helping them find weak spots or assumptions. However, they also found some challenges, like missing subtle problems hidden by the simulation or relying too much on generated themes.
What this means in practice
- •For qualitative research teams: Use simulated participants to practice and improve interview questions before meeting real participants, increasing interview quality.
- •For user experience researchers: Employ simulated interviewees to refine user study protocols and identify gaps in target populations more efficiently.
Authors
Hangyue Zhang, Qiran Hu, Ziyi Zhang, Hyanghee Park, Yun Huang
Abstract
LLMs are increasingly used across research workflows, including research ideation, qualitative analysis, and participant simulation. Within work on participant simulation, prior studies have largely evaluated simulated participants by how realistic or human-like their responses appear, leaving less known about how researchers use and critique these materials when revising interview protocols before fieldwork. We present Rehearsim, an inspectable research probe that treats LLM-simulated participants as materials for methodological rehearsal rather than substitutes for human participants. Rehearsim combines literature-grounded participant construction, interactive interview rehearsal, and traceable thematic review while keeping protocol decisions under researcher control. In a study with 21 qualitative researchers, participants used simulated materials to surface assumptions and gaps, reconsider participant populations, revise target populations, and develop follow-up probes. They also identified risks, including the concealment of weak questions, dismissal of unexpected synthetic details, and interpretive anchoring from generated themes. We derive an inspectable-rehearsal framework emphasizing traceability, contestability, researcher control, and epistemic boundaries.