LLM tutors can improve learning but require careful setup

Do LLM Tutors Actually Teach? A Process-Centered Synthesis of the Tutoring Loop

Computation and Language

Summary

LLM tutors can generate fluent tutoring conversations, but it's unclear how much they actually help students learn. The authors analyzed how these tutors work from detecting what learners know to taking teaching actions, training the tutors, testing, and simulating students. They found a key issue: just building a tutor function doesn’t prove it truly helps learning or influences later steps. Studies show that LLM tutors can help, but success depends on how the tutoring is structured, how engaged learners are, and how learning is measured. The authors suggest future work should check each step’s connection, not just improve single parts.

What this means in practice

A survey. It maps existing work.

Authors

Sayantan Adak, Rima Hazra, Koyel Mukherjee, Digbalay Bose, Somnath Banerjee

Abstract

Large language models can produce fluent tu- toring dialogue, but whether they actually help students learn remains unclear. We present a process-centered synthesis of LLM tutoring that follows the path from learner-state infer- ence to pedagogical action, tutor training, eval- uation, and student simulation. This view ex- poses a central gap: implementing a tutoring component does not establish that it works or that its outputs meaningfully affect later stages. A structured study-level audit shows that peda- gogical evaluation is common, while evidence becomes much thinner for learner-conditioned intervention, human learning gains, and espe- cially transfer or delayed retention. Human studies nevertheless provide a conditional posi- tive result: LLM tutors can improve learning, but benefits depend on pedagogical structure, learner engagement, and how learning is mea- sured. We argue that progress requires validat- ing the interfaces between tutoring stages, not only improving individual components.
View PDFOpen arXiv