Guided ai tutors vary widely in helping students stuck on problems
Examining Variation in How Guided AI Tutors Resolve Student Impasses
Artificial Intelligence
Summary
When students get stuck on a problem, AI tutors face a tricky choice: help too soon might stop students from learning deeply, but waiting too long can leave them frustrated. The paper studies how an AI chemistry tutor behaves when students face three types of sticking points: wrong ideas, uncertainty, or asking for help. The authors found that giving the answer right away happened often unless the AI was told never to do so. Asking guiding questions helped at first but became less useful the longer a student stayed stuck. Instead, directly addressing students' errors helped them get unstuck more often over time.
What this means in practice
- •For edtech developers: Design AI tutoring systems that adjust help based on the type and persistence of student difficulties to improve learning outcomes.
- •For online chemistry tutors: Implement dialogue prompts that shift from questions to error explanations as students remain stuck, aiding faster problem recovery.
Authors
Bakhtawar Ahtisham, Kirk Vanacore, Alessandra Napoli, Josh Arens, Ksenia Ionova, Clayton Cohn, Shima Salehi, Rene Kizilcec
Abstract
When a student is stuck, a tutor faces the assistance dilemma: help given too early can hinder productive struggle, while help withheld too long leaves the student in a frustrating, persistent impasse (i.e., wheel spinning). Generative AI tutors increasingly use guardrails restricting answer-giving, yet little is known about how such tutors behave once an impasse persists. We analyze 20,462 student turns from 1,260 authentic sessions with a guided LLM chemistry tutor, identifying 6,630 impasse turns of three major types: conceptual errors, expressed uncertainty, or help-seeking. We then used these impasses to simulate three tutoring conditions to study variation in AI tutor guidance through impasses: baseline, no-direct-answer, and guided tutor. For a sample of 150 impasses, prompt specificity changed pedagogy: a baseline tutor provided the answer directly in 50.7% of responses, a no-direct-answer tutor asked a follow-up question every time, and the guided tutor responded in a wide variety of ways depending on the context. We then analyzed impasse trajectories in authentic interactions, finding that each additional impasse turn lowered the odds of next-turn recovery by 12.7% (AOR = 0.873, p < .001), and early dropouts were caught in recursive concept elicitation before reaching execution. The benefit of questioning decayed as impasses persisted (scripted question x depth AOR = 0.78; follow-up x depth AOR = 0.83), whereas addressing the student's error grew more beneficial (AOR = 1.14); after a failed scripted question, repeating it was followed by recovery in 28.1% of cases, compared with 39.8% when the tutor addressed the error instead. For learning analytics, these findings identify impasse depth and type as observable, turn-level dialogue signals that analytics can use to trigger graduated, state-sensitive assistance in real time.