Student behavior modeling improves simulated tutoring interactions
HACHIMI-II: From Role-Play to Learner Behavior Modeling for Faithful Student Simulation
Computation and LanguageArtificial IntelligenceComputers and SocietyMachine Learning
Summary
Simulated students can help test tutoring systems before real students try them, but usual methods just make the simulation act like a generic student, not a real one. The authors created HACHIMI-II, which looks at what a real student has done before and tries to predict their next action realistically. This makes the simulation behave more like actual students, including making similar errors. This helps researchers test tutoring methods more accurately.
What this means in practice
- •For educational software developers: Create tutoring systems that can be tested with student-like simulations reflecting real learners' successes and mistakes.
- •For ai dialogue system designers: Build student simulators that replicate realistic student dialogue errors to improve tutoring conversation models.
Authors
Yilin Jiang, Fei Tan, Xuanyu Yin, Jing Leng, Aimin Zhou
Abstract
Simulated students let researchers test tutoring systems and teaching strategies before real learners are available, but only if the simulation reproduces where a particular learner succeeds and fails. The common approach asks a large language model to role-play a student, but a role does not model what this learner knows or will do next, so the model's own competence fills the gap. Simulated students perform at a level set by the model, differ little from one another, and seldom make the real learners' errors. We model the learner's behavior and recast student simulation as third-person prediction of the learner's next action. HACHIMI-II builds a student model from the learner's history, infers the learner's state for the current item or tutor turn, and compiles it into a behavior harness that constrains the next action. On three item-response datasets, the Wasserstein-1 distance between simulated and real student ability is 2.4--8.4$\times$ smaller than that of the strongest baseline. On two dialogue datasets, HACHIMI-II reproduces real student errors with the same knowledge type, locus, and mechanism 1.8--5.1$\times$ more often than the strongest baseline, and ablations trace the gains mainly to the explicit learner state. Predicting the learner makes capable models usable as simulated students.