TeachGrade improves short answer scoring accuracy without retraining models
TeachGrade: Training-Free Short Answer Scoring with Teacher Evidence
Computation and Language
Summary
Grading short answers fairly is tricky when different teachers have different standards. The authors created TeachGrade, a way to grade answers by using examples of teacher-graded work instead of training new models. Their approach compares new answers to these examples to give fair scores that align with teacher preferences. They tested TeachGrade on thousands of answers across many tasks and languages, showing it scores more consistently and accurately than existing methods without needing to change the AI itself.
What this means in practice
- •For education technology teams: Improve automated grading systems by incorporating teacher examples without retraining language models, enabling consistent and fair short answer scoring.
- •For multilingual platform developers: Deploy scoring tools that adapt to different languages and grading styles using the same pretrained models and teacher data, without extra model training.
Authors
Jinhao Yan, Zhihong Shen, Shengjie Wang, Yiting Ren, Wentao Zhang, Yilei Wang, Peichao Lai
Abstract
Multi-subject, multi-step scoring requires consistent partial credit across grading conventions. We introduce TeachGrade, a training-free framework using teacher-graded examples. The Task Adapt module builds two frozen memories: Skill captures contextual teacher preferences beyond the rubric; CaseBank supplies local evidence. PreJudge provides retrieval cues; FinalJudge regrades the answer without preliminary scores. On 3,829 SAS-Bench answers across 12 tasks, official macro collaborative consistency score (CCS) rises from Native's 67.99 to 81.32 (+13.33 points), and pooled step exact match from 54.85% to 62.71%. Macro quadratic weighted kappa (QWK) reaches 84.71, exceeding Native, OpenJudge, ProofGrader, and Trojsten by 9.52, 3.50, 11.37, and 17.43 points, respectively, and leading on 8 tasks. Local Qwen3.8-27B gains 8.83 CCS and 4.08 QWK points, improving CCS on all 12 tasks. QWK gains over Native span 6.38--14.65 points on Portuguese automatic essay scoring (AES) and 2.16--2.24 on ASAP-SAS across label views, with lower mean absolute error. These results support reuse across languages, response formats, and backbones without weight updates.