Papers for

education technology teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

User interface design affects challenges in AI oversight work

What UI Design Can Do For "Good" Oversight Work: A Three-Month Action Research Study

Abstract: With AI automating consequential tasks, human oversight is required for ethical, legal, and practical reasons. Yet these oversight tasks can be demanding and often make for poor work. Empirical evidence on overseers' lived work experiences remains underexplored, yet critical for designing ``good'' oversight. We provide longitudinal, in-the-wild insights through action research accompanying (N=8) tutors who oversaw an AI-supported learning assistant over three months. Our thematic analysis combines insights from twelve weekly meetings and two workshop iterations and was enriched by four repeated questionnaires. We identified stimulation, mastery, and relatedness as key challenges, stemming from the task's repetitiveness, unclear impact of interventions, and limited connections with students and peers. Informed by our initial findings, we deployed targeted UI modifications with mixed success. This paper documents the long-term evolution of the work characteristics of a real-world oversight task and discusses which oversight work characteristics can and cannot be supported by interface design.

Mon 12 OctHuman-Computer Interaction
The gist
When people oversee AI systems doing important tasks, the oversight work can be hard and repetitive. The authors studied tutors who supervised an AI assistant over three months to learn about their daily struggles. They found the work felt boring, unclear in impact, and isolating, which made it tough to stay motivated. The authors tried changing the user interface to help, but it only partly improved the experience. Their study shows how interface design can support some parts of oversight but not all the challenges.
Open → 2610.16410v1

TeachGrade improves short answer scoring accuracy without retraining models

TeachGrade: Training-Free Short Answer Scoring with Teacher Evidence

Abstract: Multi-subject, multi-step scoring requires consistent partial credit across grading conventions. We introduce TeachGrade, a training-free framework using teacher-graded examples. The Task Adapt module builds two frozen memories: Skill captures contextual teacher preferences beyond the rubric; CaseBank supplies local evidence. PreJudge provides retrieval cues; FinalJudge regrades the answer without preliminary scores. On 3,829 SAS-Bench answers across 12 tasks, official macro collaborative consistency score (CCS) rises from Native's 67.99 to 81.32 (+13.33 points), and pooled step exact match from 54.85% to 62.71%. Macro quadratic weighted kappa (QWK) reaches 84.71, exceeding Native, OpenJudge, ProofGrader, and Trojsten by 9.52, 3.50, 11.37, and 17.43 points, respectively, and leading on 8 tasks. Local Qwen3.8-27B gains 8.83 CCS and 4.08 QWK points, improving CCS on all 12 tasks. QWK gains over Native span 6.38--14.65 points on Portuguese automatic essay scoring (AES) and 2.16--2.24 on ASAP-SAS across label views, with lower mean absolute error. These results support reuse across languages, response formats, and backbones without weight updates.

Sat 10 OctComputation and Language
The gist
Grading short answers fairly is tricky when different teachers have different standards. The authors created TeachGrade, a way to grade answers by using examples of teacher-graded work instead of training new models. Their approach compares new answers to these examples to give fair scores that align with teacher preferences. They tested TeachGrade on thousands of answers across many tasks and languages, showing it scores more consistently and accurately than existing methods without needing to change the AI itself.
Open → 2610.14900v1

Bengali vision language models struggle with geometry diagrams and text

ChitraMiti: Benchmarking Visual Grounding and Modality Reliance in Bengali Geometric Reasoning

Abstract: Evaluation of vision-language models (VLMs) for multimodal mathematical reasoning remains limited for low-resource languages and for geometry problems that require reading a diagram and a question together. We introduce ChitraMiti-12.8k, a synthetic benchmark of 12,874 Bengali planar geometry problems paired with structured 15-attribute descriptions, and NCTB-500, a complementary set of 500 diagrams manually extracted from Bengali school textbooks. Using a three-phase protocol that separates diagram-only, diagram-plus-description, and description-only inputs, we show across five open-weight and closed-source VLMs that description-only performance is statistically indistinguishable from diagram-plus-description performance, establishing structured descriptions as a sufficient textual proxy for controlled evaluation. Despite this, models remain poor at cross-modal verification, frequently misled by a swapped spatial relation even when they answer the unmodified item correctly. We further evaluate supervised adaptation on ChitraMiti-12.8k, finding that fine-tuning improves performance on both ChitraMiti-1k and NCTB-500, although a substantial gap to the strongest zero-shot model remains. Together, ChitraMiti-12.8k, NCTB-500, and our evaluation protocol offer a standardized way to study Bengali multimodal geometry reasoning and, more broadly, whether VLMs actually check their text against what they see. Our dataset and code are publicly available on Hugging Face at https://huggingface.co/datasets/RaiyanKhaan/ChitraMiti.

Fri 11 SeptComputer Vision and Pattern Recognition
The gist
It is hard for computer models that understand both images and language to solve math problems in Bengali, especially those involving geometric diagrams. The authors created a large set of Bengali geometry problems with both pictures and detailed text descriptions to test these models. They found that models do about the same when given only text descriptions as when given both images and text, showing the descriptions can stand in for diagrams in tests. However, models often fail when asked to confirm that the text matches the image, mixing up spatial relationships. Training the models on this dataset helps, but they still lag behind the best models used without extra training. Overall, this work gives a new way to study how models handle math problems combining images and language in Bengali.
Open → 2609.12509v1