Students show limited critical questioning when using AI in math chats
Student-LLM Interactions During Math Learning: Limited Epistemic Proactivity Overall, but Within-chat Change Predicts Performance Without AI
Computers and Society
Summary
The paper studies how secondary-school students interact with AI tools when solving math problems. It finds that students mostly ask the AI for answers without much planning or checking. However, students who start asking better questions over time do better on later math tests without AI help. This suggests that paying close attention to how students use AI during learning is important for understanding their real progress.
What this means in practice
- •For educational technology developers: Develop AI tutoring systems that monitor and encourage students’ critical questions during math problem chats to boost learning outcomes without ongoing AI help.
- •For online course designers: Design math learning platforms integrating AI that track how students’ question styles evolve to predict and support improved post-AI test performance.
Tested on one dataset.
Authors
Rania Abdelghani, Peter Kaiser, Kou Murayama
Abstract
Large language models (LLMs) are increasingly used for learning, raising a key question: can learners remain epistemically proactive, retaining responsibility for the cognitive and regulatory activities through which learning is produced while working with AI? Because this responsibility may change as interactions unfold, we examine epistemic proactivity as a developing process. We conceptualize epistemic proactivity through self-regulated learning (SRL) moves, complemented by help-seeking strategies and manifestations of epistemic vigilance. We analyzed these behaviors in 880 substantive turns from 112 secondary-school students completing open-ended mathematical-modeling practice with a general-purpose LLM (Mistral). Overall, students showed limited observable epistemic proactivity: their interactions were strongly request-dominated, with planning, monitoring, and evaluation relatively rare, although low-quality AI responses elicited more explicit evaluation. Temporal analyses qualified this picture: explicit SRL-related moves increased modestly over the chat, but help-seeking strategies became less understanding-oriented. Crucially, these temporal trajectories in help-seeking strategies predicted their post-test performance without AI beyond pre-test and aggregate patterns: students whose requests became more understanding-oriented performed better once AI assistance was removed. Thus, while students had a limited epistemic proactivity overall, its temporal development still was a critical predictor of post-test performance without AI. These results underscore the importance of examining the temporal organization of epistemic proactivity, rather than isolated prompt types, for understanding AI-supported learning. We release our fully-anonymized dataset publicly in an OSF repository.