LLM simulated clients show potential in online counselling training
Comparing Human Role-Players and LLM-Simulated Clients in Online Counselling Training: An Analysis of Counselling Patterns
Artificial IntelligenceComputation and Language
Summary
Training counselling skills usually involves practicing with real people acting as clients. This study compared those real role-players with computer-generated clients made by a large language model. The authors found that the AI clients can mimic some conversational patterns well but differ in certain ways from humans. They also discovered that counsellors change how they communicate depending on whether the client is human or AI. This suggests AI could help supplement counselling training but still needs improvements.
What this means in practice
- •For counselling trainers: Use LLM-simulated clients to practice counselling skills when human role-players are unavailable or for additional training scenarios.
- •For virtual assistant developers: Improve chatbot designs by analyzing conversational patterns where AI simulates human roles in complex dialogues like counselling.
Authors
Eric Rudolph, Philipp Steigerwald, Jens Albrecht
Abstract
This study investigates the capabilities of Large Language Models to simulate counselling clients in educational role-plays in comparison to human role-players. Initially, we recorded role-playing sessions, where novice counsellors interacted with human peers acting as clients, followed by role-plays between humans and clients simulated by Mistrals Mixtral 8x7b using 4-bit quantization. These interactions were analysed with a counselling communication pattern system at sentence level. We investigated two key questions: (1) to what extent LLM-generated responses replicate authentic conversational dynamics and (2) whether counsellors' communication behaviour differs when interacting with human versus LLM-simulated clients. The findings highlight both similarities and differences in the application of counselling patterns across scenarios, showing the potential of LLM-based role-playing exercises to enhance counselling competencies and to identify areas for further refinement in virtual client simulations.