Llm interventions reduce persona drift in simulated conversations
Prompting Against Persona Drift: Comparing Intervention Timing and Content in LLM-Simulated Conversations
Human-Computer Interaction
Summary
Simulating student personalities with large language models (LLMs) helps test educational systems, but the personalities can shift over long chats, making simulations less reliable. The authors tested different ways to keep these personas consistent by sending reminders or instructions during conversations. They found that giving behavior-specific instructions worked best at reducing this shift, but none of the methods completely stopped the problem. Deciding what corrections to make appears more important than when to apply them.
What this means in practice
- •For educational technology developers: Maintain consistent student personas during automated testing of educational tools using targeted prompt interventions in LLM conversations.
- •For interactive chatbot builders: Improve chatbot persona stability across extended interactions by applying behavior-specific instructional prompts detected via monitoring.
Authors
Nicolas Leins, Jennifer Haase, Varvara Geronimus, Jana Gonnermann-Müller, Sebastian Pokutta
Abstract
Simulating student personas with large language models (LLMs) enables scalable evaluation of educational systems. However, behavioral drift, a progressive decline in persona consistency, can emerge over extended conversations, limiting the validity of such simulations. We evaluate five prompt-level mechanisms using separate monitoring and intervention pipelines. Across 1,200 28-turn conversations spanning four LLMs and two ADHD persona intensities, we varied when to intervene (static vs. adaptive) and what to inject (reinjection vs. reflective reminder), plus a novel adaptive condition in which a monitor generates behavior-specific instructions. Relative to no intervention, reinjection reduced the modeled rate of LLM-rated drift by 35--38\%, reflective reminders by 22--27\%, and behavior-specific instruction by 87\%. None eliminated drift. We found no evidence that adaptive timing outperformed static scheduling. Monitoring therefore appears more useful for deciding \textit{what} to correct than \textit{when} to intervene, although behavior-specific instruction requires component-level testing.