Llms struggle to adapt tutoring to students with challenging behaviors
TutorCaseBench: A Case-Grounded Benchmark for Evaluating LLM Tutors in Supporting Students with Challenging Behaviors
Computation and LanguageHuman-Computer Interaction
Summary
Many AI tutors find it hard to help students who have learning or behavior challenges because these students act in complex ways. The authors created TutorCaseBench, a test made from real student cases that checks how well AI tutors respond to such difficult situations. They tested 11 popular language models and found that while the models can recognize helpful strategies, they often fail at giving good, specific advice. This shows current AI tutors need more work to handle real-life teaching challenges well.
What this means in practice
- •For educational software developers: Evaluate and improve AI tutoring systems for students with special learning and behavioral needs using case-grounded scenarios.
- •For instructional designers: Develop better tutoring strategies by analyzing AI model performance on realistic student challenge cases.
Authors
Yeon Su Park, Chanyoung Park, Tae Soo Kim, Seungju Kim, Hyunwoo Kim, Daho Jung, Jinsung Yoon, Juho Kim
Abstract
LLMs are increasingly used as AI tutors, yet existing benchmarks overlook the challenging behaviors common among students with learning and behavioral difficulties. We introduce TutorCaseBench, a benchmark that grounds AI-tutor evaluation in qualitative case studies from pedagogy, educational psychology, and special education. Our pipeline converts documented student categories into case-specific scenarios, tutor-student dialogues, and rubrics grounded in source evidence. The benchmark contains 923 dialogues across five challenge types, evaluated through open-ended response generation (1,560 items) and multiple-choice selection (2,537 items). Across 11 state-of-the-art LLMs, even the strongest model satisfies a rubric criterion in only 55.8% of open-ended cases, while models are better at selecting appropriate strategies than generating situated responses. These results reveal a gap between intervention recognition and context-specific adaptation, especially for open-weight models. All data are publicly available at https://github.com/kixlab/TutorCaseBench.