Ai tutoring platform improves feedback on handwritten math work
Exploring Visual Access and Proactive AI Tutoring for Handwritten Mathematics
Computers and SocietyHuman-Computer Interaction
Summary
People often show their math thinking by writing by hand, which is hard to see with normal computer tutoring. The authors made a tool where a phone camera lets an AI tutor watch learners solve calculus problems on paper. This visual help made the AI check students’ work more closely and give helpful hints when it noticed mistakes. Learners liked getting feedback based on what they actually wrote, especially when the help was proactive and pointed out errors early.
What this means in practice
- •For mobile app developers: Create apps that use phone cameras to recognize handwritten math and deliver timely, personalized AI tutoring messages to learners.$Commercial implications: Enables sale of tutoring apps that uniquely integrate live handwriting input with AI feedback for improved math learning experiences.
- •For online education platforms: Integrate visual AI tutoring systems to monitor and support students’ handwritten problem solving in live or asynchronous math lessons.
Authors
Nachiket Kotalwar, Conrad Borchers, Shubham Gandhi, Partha Sarathi Purkayastha, David Yaron, Carolyn P. Rosé
Abstract
Handwritten work captures mathematical reasoning that learners may not express in typed or voice-dictated messages. We built a multimodal AI tutoring platform in which a phone camera gives the tutor a live view of learners' work on paper. In a randomized online study, 117 adults solved calculus problems with Text-Reactive, Visual-Reactive, or Visual-Proactive assistance. Analysis of 2,140 tutor messages showed that visual access shifted assistance toward checking ongoing work, while proactivity enhanced error identification and directive hints. Learners valued feedback grounded in their writing; those who received proactive help rated it as supportive, and about a third wanted it more often. These findings inform when tutors should intervene and what they should address. We release the platform, prompts, and instruments, together with 255,984 timestamped images of handwritten work linked to tutoring conversations, intervention decisions, and learning outcomes, to support multimodal tutoring systems that preserve the affordances of handwritten work while providing adaptive feedback.