The Correct Answer Trap: Pedagogically-Grounded Detection and Feedback for Hidden Misconceptions
2026-06-22 • Computers and Society
Computers and SocietyArtificial IntelligenceInformation Retrieval
AI summaryⓘ
The authors studied how automatic feedback systems can miss hidden student misunderstandings when the final answer is correct but the reasoning is wrong. They tested different models and found that while some could detect these errors better than others, many false alarms still happened. To improve this, the authors suggest separating the evaluation of the answer from the method used, and created a system that asks follow-up questions when unsure instead of immediately alerting teachers. They also designed two ways to use this system: one to help teachers review answers more easily and another that lets an automated tutor handle simpler cases.
automated feedbackhidden misconceptionsmachine learning classifiersreasoning modelfalse alarmsassessment rubricdiagnostic questionsteacher dashboardautonomous tutorformative assessment
Authors
Moiz Imran, Sahan Bulathwela
Abstract
Automated feedback systems that rely on answer correctness will reinforce, rather than address, misconceptions when students reach the correct answer through flawed reasoning. We investigate automatic detection of these hidden misconceptions using 20,964 real student responses from the Eedi mathematics platform. Fine-tuned classifiers detect only 57% of these hidden misconceptions, and standard ML interventions do not improve on this. An open-weight reasoning model detects 84%, but at realistic prevalence, false alarms outnumber genuine detections roughly 8 to 1. We present a graduated assessment rubric that separates answer correctness from method validity, and propose a detect-verify-escalate pipeline that routes uncertain cases to diagnostic follow-up questions rather than directly to teachers. Two deployment modes adapt the pipeline: a teacher dashboard where the system filters a review queue, and an autonomous tutor where flags trigger low-cost formative follow-up.