Large language models enhance analysis of student science discussions
Modeling Student Sensemaking with LLMs and Knowledge-Graph-Guided Inference
Computation and Language
Summary
Understanding how students talk and figure out science problems together is hard and takes a lot of expert time. The authors tested whether certain language AI models can help analyze these group talks without needing special training for the task. They found that models given some reasoning help detect when students are struggling more accurately. Adding info about students’ current knowledge also made the models better at spotting difficulties. However, no single setup was best in all ways, showing this analysis is complex.
What this means in practice
- •For edtech developers: Integrate language models to automatically identify when learners struggle in science group discussions to provide timely support.
- •For learning platform teams: Use knowledge-graph-guided models to improve automated feedback systems that detect and address incomplete student explanations.
Authors
Özge Alacam, Zübeyde Demet Kirbulut Güneş, Funda Ekici, Nurcan Turan-Oluk, Dilay Dinçdemir, Hakkı Kadayıfçı, Sevinç Nihal Yeşiloğlu, Burcu Işık, Halil Tümay, Sinem Gencer
Abstract
Collaborative science learning requires nuanced interpretation of student dialogue to characterize how learners identify knowledge gaps, build explanations, and work toward resolution - a theory-driven analysis that is labor-intensive and difficult to scale. We investigate whether instruction-tuned large language models (LLMs) can support multidimensional analysis of collaborative sensemaking without task-specific training, and whether structured knowledge-state information improves model inference. We evaluate two mid-size LLMs on 23 richly annotated, expert-labeled episodes across prompting conditions that vary definitional scaffolding, reasoning mode, and turn structure. Without reasoning, models tend to overpredict successful sensemaking; reasoning-enabled prompting improves identification of unsuccessful cases. Knowledge-state diagnostics provide additional grounding, improving detection of unsuccessful sensemaking and increasing agreement with expert annotations. No single configuration performs best across all sensemaking dimensions, underscoring the multidimensional nature of the task.