online education platforms

Abstract: Handwritten work captures mathematical reasoning that learners may not express in typed or voice-dictated messages. We built a multimodal AI tutoring platform in which a phone camera gives the tutor a live view of learners' work on paper. In a randomized online study, 117 adults solved calculus problems with Text-Reactive, Visual-Reactive, or Visual-Proactive assistance. Analysis of 2,140 tutor messages showed that visual access shifted assistance toward checking ongoing work, while proactivity enhanced error identification and directive hints. Learners valued feedback grounded in their writing; those who received proactive help rated it as supportive, and about a third wanted it more often. These findings inform when tutors should intervene and what they should address. We release the platform, prompts, and instruments, together with 255,984 timestamped images of handwritten work linked to tutoring conversations, intervention decisions, and learning outcomes, to support multimodal tutoring systems that preserve the affordances of handwritten work while providing adaptive feedback.

Abstract: Timely pedagogical feedback is difficult to scale in online classes, where long and noisy videos make student responses hard to locate and automated systems often produce generic advice weakly grounded in actual classroom behavior. The key challenge is to connect the instructional goal of a specific lesson segment with temporally aligned multimodal evidence of student responses, and then translate the observed gap into an actionable teaching move. We introduce MAVE (Multi-Agents for Video-based Educational Feedback), a multi-agent framework for grounded pedagogical feedback from online-class videos. MAVE decomposes feedback generation into four roles: the Curator structures the lesson and infers expected student activity; the Observer extracts aligned speech, visual, and interaction evidence; the Assessor estimates the activity supported by student behavior; and the Advisor generates gap-conditioned teaching recommendations. To evaluate this capability, we introduce MAVEBench, a benchmark designed to measure whether feedback systems can ground diagnoses in multimodal student evidence, and generate actionable pedagogical feedback. MAVEBench contains 173 knowledge-point segments from approximately 20 hours of authentic online-class videos across STEM and Humanities, together with comprehensive simulation. Experiments show that MAVE outperforms direct prompting, Reflexion, and TextGrad in diagnostic correctness, instructional alignment, and actionability. Our code is available at https://github.com/DaRL-GenAI/MAVE

Abstract: Adaptive assessment systems personalize learning by estimating learner proficiency and recommending suitable questions. However, many existing approaches rely on fixed difficulty labels, static rating updates, or black-box prediction models that provide limited control over rating stability, interpretability, and time-constrained routing. This paper proposes MARS, a Momentum-Aware Adaptive Rating System for interpretable adaptive assessment. MARS represents the question bank as a directed weighted knowledge graph and updates learner proficiency using dynamic rating sensitivity, response quality, and momentum-aware learner velocity. We also formulate a time-budgeted mark-maximization objective in which the policy selects questions within a fixed test duration under negative marking. Experiments on ASSISTments 2009 and MARS-SimExam evaluate MARS for prediction, rating stability, and adaptive routing. Results show that MARS provides smoother rating updates than Fixed Elo and achieves the highest score under time-budgeted negative-marking routing, improving the total score by 8.09 over the strongest baseline and reducing overtime violations from 50% to 5.6%. These results suggest that MARS is suitable for real-time, interpretable, and deployment-friendly adaptive assessment.

Abstract: AI-text detectors are already applied to Kazakh writing, yet no study reports how often publicly available detectors flag Kazakh text whose human authorship is dated before ChatGPT. We present AINA-KK: 3,811 human passages from encyclopedic, news, web and educational pages that provably existed before ChatGPT, and 5,597 machine and gray-zone passages from 9 generators, including the Kazakh-built KazLLM and Sherkala, with script and length stress tests. Five English-trained detectors from the Hugging Face Hub flag up to 81.4% of human passages and 79.6% of published student essays; recalibrated, they catch at most 1.3% of machine text. Calibrated zero-shot detectors catch at most 50.3% of machine text from in-distribution generators. Fine-tuned encoders rank text almost perfectly, but mDeBERTa at its default threshold flags 24.4% of 50-word human excerpts, partly because machine outputs are shorter than human text; calibrated, it catches only 47.1% of KazLLM and 62.7% of Sherkala passages. Without script handling, machine-transliterated Latin-2021 text defeats every detector. Fixes that need no new data (calibration, script canonicalization, script-augmented and length-matched training, and an OR with a zero-shot scorer) form AINA-KK-Det. On test it flags 0.9% of human passages (95% CI 0.4-2.7; 1.5% on development data) and catches 85.2% of machine text from known generators, 92.9% from two unseen Gemini-family Gemma-4 models and 61.3% from the Kazakh-built ones; its configuration was chosen with test results in view. We release the data, code and detector at https://github.com/ozxmn/aina-kk.

Abstract: Real-time classroom polls are widely used and increasingly generated with automated assistance, yet the questions themselves are rarely evaluated as measurements. We audit a large corpus of authentic classroom polls, 604 items across 47 sessions answered 340,668 times by 2,807 learners, as a measurement instrument. For the 539 items whose correct answer could be established and verified from the lecture transcript, we place every item and every student on a common scale using item response theory and analyse the answer structure of the True/False items. Two findings emerge. First, the polls form a coherent but easy scale of moderate precision (marginal reliability about 0.60), on which roughly a quarter of items barely separate stronger from weaker students. Second, students show a robust tendency to answer True, present at the individual level (77% of students lean True), which meets a milder tendency for items to be keyed False; as a result answer direction predicts difficulty, False-keyed items being about thirteen points harder, and the effect survives controls for item content and for selective answering. Both findings rest on signals a polling system already records, so the same checks can be run as items are generated, before they reach students.

Abstract: Knowledge concept tagging aims to assign specific concept or topic labels to educational content, which is essential for both educators and learners in traditional and online teaching practices. Recent work has explored large language models (LLMs) for this task, achieving promising performance. However, LLMs still struggle to select the correct concept from a large-scale candidate set due to the high dimensionality of the decision space. In this paper, we propose a novel three-stage Select-Reason-Judge (SRJudge) framework, which empowers LLMs with selective reasoning capability for fine-grained knowledge concept tagging. Specifically, the Selector in Stage 1 first narrows the candidate concepts to a top-K shortlist by fine-tuning a small language model (SLM), e.g., BERT, since the top-$K$ predictions hit the correct concept in most cases, thereby reducing the decision space of correct candidates. Next, the Stage 2 Reasoner employs a lightweight LLM for refined reasoning over the shortlisted candidates. It further integrates an improved reinforcement learning strategy with a dynamic task-specific reward function and a pruning mechanism to better align with human reasoning preferences. Finally, a larger LLM acts as a judger that evaluates the overall rationality of the reasoning process and its explanations to determine the final output. In addition, we construct two high-quality datasets for further validation, i.e., the biology dataset S_Bio and the physics dataset S_Phy. Experimental results demonstrate that our method consistently outperforms state-of-the-art baselines across benchmark datasets, verifying its effectiveness and superiority. Resources are available at: https://github.com/Nicozwy/SRJudge.

Abstract: Artificial intelligence offers an unprecedented opportunity to augment human capabilities, yet progress at the frontier has focused primarily on advancing model capabilities. We introduce StudentBench, a suite of AI teaching evaluations and a public platform that enables large-scale data collection with over 175,000 student-AI messages to study whether large language models (LLMs) produce learning gains equivalent to human tutoring. Using StudentBench, we measured learning gains on Quantitative and Verbal GRE questions across 2,383 human participants receiving AI tutoring, human tutoring, or no tutoring. We establish that AI tutoring is statistically equivalent to expert human tutoring for GRE learning gains (p = .015), and in five of the seven GRE domains, the best performing AI tutor surpassed the human tutor, on average. In a second study, expert human tutors compared LLM-generated lesson plans and practice problems through 2,028 pairwise rubric evaluations. Together, the two studies clearly separate AI tutors across: (1) lesson planning, (2) practice-problem creation, (3) conversational pedagogy, (4) cost, and (5) engagement. Surprisingly, one AI tutor achieved learning gains equivalent to human tutoring (p = .044) at 918 times lower cost (USD 0.0052 for AI versus USD 4.81 for human, per percentage point gained). For Quantitative GRE sessions, faster AI replies correlated with more student messages, more messages with more correct practice, and more correct practice with larger learning gains (all p < .002). The StudentBench platform is freely available at https://studentbench.org.

Abstract: Lifelong learners from different industries often watch the same recorded lecture, even when they will apply the material in different workplaces. We present Bespoke, a system that takes the transcript of an existing lecture and generates a new video customized to a target industry and duration, with new slides, narration, and charts. From 31 graduate lectures in analytics, machine learning, and optimization, we generated 209 videos for these three industries, plus a generic-audience version of each lecture. Twenty-five experts in the corresponding domains rated 92 videos on a five-point rubric. They judged 87% at or above the rubric midpoint corresponding to ``a standard MOOC lecture's quality'' (mean overall score 3.42 out of 5; 48% scored 4 or above). Overall quality was similar across industries, durations, and a 21-lecture held-out set unused while developing the system, at about \$0.22 API cost per minute of video.

Abstract: Durable skills, such as collaboration, creativity and critical thinking, are instrumental to success in the modern workforce. Yet, measuring these skills remains a persistent challenge. Moreover, because what is not measured is often not taught, these skills are often overlooked in mainstream educational curricula. Designing effective assessments for these skills necessitates balancing two often-conflicting requirements: ecological validity and psychometric rigor. On the one hand, the assessment environment should emulate natural real-world human interaction between humans. On the other hand, it should be scalable, controllable and reproducible. Here we argue that LLMs can be used to better capture both of these aims. Concretely, we develop a framework where the subject converses with AI teammates in a way that resembles human-human interaction for authenticity, while also offering the psychometric control required for informative and robust assessment. Importantly, the AI participants not only act as teammates but also, in an "Executive LLM" setup, steer the conversation towards eliciting a high density of observable evidence for skill proficiency. We complement this with an AI evaluator that can be used to measure skill proficiency in such interactions. We evaluate our assessment protocol based on transcripts of interactions of human participants with our AI framework, for multiple durable skills. For the skill of creativity, we further demonstrate the efficacy of an autorater for evaluating complex tasks performed by real students. Our analysis shows that the use of the Executive LLM significantly increases elicited evidence and that LLM-automated scoring of conversations largely agrees with that of expert annotators. This research demonstrates the utility of orchestrated LLMs approaches for measuring complex social and cognitive constructs in a scalable and controllable manner.