Papers for

math tutoring technology makers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Large language models spot common student errors in math modelling

Finding Common Mistakes In Modelling With Mathematical Formalisms Using LLMs

Abstract: Modelling with mathematical formalisms like logical formulas, mathematical equations, or regular expressions is an important yet challenging task for students of computer science and other STEM disciplines. Identifying common mistakes occurring in this context is an important step towards helping struggling students by providing targeted high-quality feedback, e.g. in interactive learning systems. We present a tool-supported workflow that allows to (1) identify candidates for common mistakes that explain many student mistakes in large educational data sets, (2) cluster candidates according to similarities, and (3) visualize resulting clusters for instructors and CS education researchers. The visualization is designed to help researchers to identify common modelling mistakes. The candidates for common mistakes are represented by bug fixing transformations that translate incorrect formalizations into correct formalizations; they are generated by an LLM and validated algorithmically. We show that this approach works well by reproducing common mistakes in propositional logic modelling that were identified by hand in the literature; showing that, unlike other algorithmic approaches, the LLM-based approach is suitable for very large sets of data; and applying it to multiple other formalisms to showcase it generalizes beyond propositional logic.

Tue 15 SeptComputers and SocietyArtificial IntelligenceLogic in Computer Science
The gist
Students learning computer science and related fields often make mistakes when writing math-based models like logical formulas or equations. The authors created a process where a large language model (LLM) suggests ways to fix these mistakes, groups similar mistakes together, and then shows these clusters in a visual tool. This helps teachers and researchers see common errors students make. Their method works well on big sets of data and applies to different kinds of math formalisms beyond just logic formulas.
Open → 2609.17111v1