Papers for

automated tutoring platforms

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Large language models improve math reasoning by classifying error types

Improving Mathematical Reasoning Capabilities in Large Language Models via Reasoning Process Error Classification

Abstract: The reasoning ability of large language models (LLMs) is a critical factor for practical LLM-based applications. To investigate the current reasoning capability of LLMs, we clarify the types of errors that arise in LLMs' reasoning processes on mathematical datasets. We focus on problems where LLMs produce an incorrect answer. We define errors in the reasoning process as reasoning errors and manually analyze the features of reasoning errors. We defined and classified 21 error classes and identified the frequently occurring classes among them. Beyond qualitative evaluation, we leverage the evaluation results to improve the reasoning capability. We designed a prompt that explicitly focuses on eight error classes. The experiments demonstrate that this prompt effectively improves reasoning performance. Furthermore, the results suggest that the frequent reasoning errors identified in this paper are common across LLMs of comparable scale.

Mon 14 SeptComputation and Language

The gist

Large language models (LLMs) sometimes get math problems wrong because of different kinds of mistakes in their reasoning steps. The authors studied these mistakes by defining and categorizing 21 types of reasoning errors. They found some error types appeared more often and designed a prompt to help LLMs focus on eight common errors. Using this new prompt helped the models perform better at math questions. They also noticed these common errors happened across similarly sized language models.

Open → 2609.15145v1