Unified framework predicts question difficulty and student answers in tutoring
A Unified Response Framework for Questions and Tutoring Dialogues
Computation and LanguageComputers and SocietyMachine Learning
Summary
Many educational tools focus only on how students answer questions, but tutoring often involves back-and-forth conversations that provide more clues about what a student knows. The authors developed a method that treats both questions and parts of tutoring talks the same way to better understand student learning. Their approach turns tutoring dialogue into clear questions, estimates how hard these are, and keeps track of how students improve over time. Tests on real math tutoring conversations showed this method predicts question difficulty and student answers more accurately than previous approaches.
What this means in practice
- •For educational software developers: Improve math tutoring apps by dynamically adjusting question difficulty and tracking student progress from dialogues.
- •For online tutoring platform engineers: Integrate a unified student model that analyzes dialogue subtasks to enhance personalized tutoring experiences.
Authors
Zhangqi Duan, Simon Woodhead, Maya Miller-Vedam, Andrew Lan
Abstract
Existing student models in educational applications largely focus on student responses to questions. However, as tutoring becomes more accessible at large scale, it is important to analyze dialogues between tutors and students, which may reveal fine-grained evidence on student knowledge. In this work, we propose a unified framework that models both student responses to questions and tutor-posed subtasks during dialogues on a shared item response theory (IRT) scale. We first identify mathematical subtasks in tutoring dialogues, rewrite them into self-contained questions, and identify the correctness of student responses to these subtasks. We then develop an IRT framework that estimates question/subtask difficulty from text, tracks students' evolving abilities across question-answering and dialogue interactions, and a novel method to leverages tutor moves between consecutive subtasks to regularize their difficulty values. We evaluate our framework on two real-world math education datasets containing both human- and AI-tutored dialogues. Results show that our framework outperforms strong baselines on both question difficulty prediction and response correctness prediction.