Papers for

automated tutoring developers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Internal signals improve error detection for language model answers

Auditing the Judge: Internal Signals for LLM Reasoning Assurance

Abstract: Language-model systems use self-judgment to decide which answers to release, review, or recompute. Because intervention is costly, an assurance score must rank errors well at the available budget, not merely over the full ranking. We compare the judge's continuous verdict-token score with a supervised correctness probe trained on attention-head activations from the same prompt. Exact identities link fixed-budget error recall to residual risk and show that AUROC averages recall over all budgets. Across three open 7-12B models on GSM8K and MATH, the probe finds 1.3-22.5 additional percentage points of errors at 5% review in all six model-dataset settings (median 7.4 points). At 10% routing, probe-guided allocation improves accuracy over equal-cost random allocation in both GSM8K experiments, but beats judge-guided routing only for the weaker judge. Cross-dataset transfer lowers probe AUROC for every model in both directions relative to target-domain training. Because dataset, weight precision, and generation budget change together, this measures a joint protocol shift rather than a dataset-only effect. Internal signals can improve supervised, in-domain budget allocation, but are not domain-general confidence estimators.

Tue 10 NovMachine Learning
The gist
Language models try to judge when their answers might be wrong to decide which ones need more checking. The authors studied two ways to find mistakes: one from the model’s own confidence signals and another from a probe trained on the model’s internal activity. They found that the probe can catch more errors with a fixed review budget, but it works best when trained on the same type of data. The paper shows that internal signals can help but don’t always generalize well across different tasks.
Open → 2611.14323v1

Hybrid reasoning method improves accuracy with shorter steps

Learning When to Think Aloud in Latent Reasoning

Abstract: Latent Chain-of-Thought (CoT) reasoning improves efficiency by shifting intermediate computation into continuous hidden states. However, fully latent methods often struggle on precision-critical tasks such as mathematical reasoning, while existing hybrid approaches retain substantial token-generation overhead. We introduce VITAL, an importance-guided hybrid reasoning framework that selectively verbalizes informative tokens as reasoning anchors while keeping most computation latent. A lightweight gating head dynamically selects between explicit token generation and direct hidden-state feedback. VITAL constructs hybrid trajectories from explicit CoT using token- importance estimation and trains the model through curriculum-based supervised training, followed by reinforcement learning with a reward that balances correctness and conciseness. Experiments show that VITAL consistently out- performs existing fully latent reasoning methods, achieving relative average accuracy gains of 9.6% on GPT-2 and 5.1% on Llama-3.2-1B at comparable reasoning lengths. Compared with hybrid baselines, VITAL achieves higher reasoning accuracy with substantially shorter reasoning trajectories on Qwen3-1.7B.

Wed 4 NovMachine Learning
The gist
Some computer programs solve tricky problems by thinking step-by-step out loud, but this can be slow, or by thinking silently inside their minds, which can lose accuracy. The authors created VITAL, a way for programs to decide when to say their thoughts out loud and when to keep them secret. This approach helps the program be more accurate on math problems while not taking too long. They tested VITAL on different AI models and found it works better and faster than other methods that either think all out loud or all quietly.
Open → 2611.06108v1

Supervised fine tuning data curated 100 times faster for math and physics

From Weeks to Hours: Fast and Principled SFT Data Generation for Reasoning LLMs

Abstract: Reasoning-oriented post-training enables large language models (LLMs) to solve complex tasks via multi-step inference. Supervised fine-tuning (SFT) on high-quality reasoning traces is a particularly efficient approach, but its effectiveness depends critically on careful data curation. Existing pipelines rely on a costly generate-then-filter paradigm: large pools of long reasoning traces are produced by multiple teacher models and subsequently filtered for difficulty and diversity using additional LLMs, often requiring weeks of computation. In this work, we propose SFTProbe, a fundamentally different approach that bypasses this pipeline by directly predicting the quality of reasoning data. We show that a brief LoRA-based adaptation, combined with evaluating loss on only 5% of each reasoning trace, suffices to estimate both difficulty and diversity. SFTProbe is grounded in a geometric view of post-training, where pretrained LLMs lie in a low-rank, anisotropic loss basin. By probing this basin via low-rank perturbations, we construct compact loss signatures whose clustering captures diversity, while loss at curvature inflection points provides a robust proxy for difficulty. We also provide a simple criterion for teacher selection. Empirically, our approach reduces data curation time from weeks to hours while maintaining or improving performance. Remarkably, on mathematical reasoning, 5k generated examples match the highly filtered 89k-example in OpenThoughts-Math dataset, and 7.5k generated examples outperforms it by 2.5% in accuracy. We further introduce a high-quality physics reasoning dataset, and use it to train a model that outperforms an IPhO 2025 silver-medal-winning model. The code and the dataset are released at https://bigml-cs-ucla.github.io/SFTProbe-page/.

Sat 31 OctMachine Learning
The gist
Large language models can solve complex problems by thinking through multiple steps, but teaching them this skill takes a lot of carefully chosen teaching examples, which can be expensive and slow to prepare. The authors designed a new method called SFTProbe that quickly predicts the quality of these teaching examples without generating and checking huge amounts of data. Their approach cuts down the time for preparing training data from weeks to just hours, while still improving or matching previous results in math and physics problem-solving. They also created a new physics reasoning dataset that led to a model beating a top competition silver medalist.
Open → 2611.00554v1

Language model selection method improves reasoning and math performance

Selection Is Not Learning: Choosing Model Futures

Abstract: Self-evolving language models train on tasks they generate, so every round must decide which updated model the next round inherits. Current loops such as R-Zero, Agent0, and Skill Self-Play make this decision through the task, admitting tasks that look learnable and inheriting every resulting update. We identify a selection-to-update gap in these loops, where a task that looks learnable need not produce a useful update. Most updates on verified tasks fail to make correct answers more likely, yet these loops inherit them all. HeirCommit freezes the updated model and uses fresh evaluation questions to decide whether to inherit it or retain the current model. EvoHeir separates parameter inheritance from experience inheritance, using current and archived experience to repair failures and preserve successes before final confirmation. Our analysis characterizes selection-induced optimism and decomposes regret into selection, absorption, and confirmation errors. HeirCommit cuts the harmful share of accepted updates by 68-83% relative to random inheritance at the same acceptance rate. Across four backbones, EvoHeir achieves the best mathematics and general-reasoning averages, raising the math average by 5.3% relative to the strongest baseline, Agent0. The framework lets useful experience persist even when the model that produced it is not inherited.

Thu 15 OctMachine LearningArtificial IntelligenceComputation and Language
The gist
Choosing which updated language model to keep as it learns tasks can be tricky, because some updates may not actually help. The authors show that many models inherit bad updates when picking new versions based only on learnable-looking tasks. They introduce two methods, HeirCommit and EvoHeir, which carefully test and confirm updates before accepting them, helping keep better knowledge while discarding harmful changes. This approach boosts performance on math and reasoning tasks compared to previous ways of selecting model updates.
Open → 2610.22029v1

Improving reasoning models by learning from nearby expert suggestions

Better Supervision Is Nearby: Neighborhood On-Policy Self-Distillation

Abstract: On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises student-sampled prefixes. Standard OPSD uses one fixed parameter setting at every state, but nearby settings may offer additional supervision. We find that local parameter perturbations reveal complementary reference-aligned corrections under the same reference context. Different experts supply these corrections at different reference positions. Their pool covers more such positions than the unperturbed privileged teacher. We introduce Neighborhood OPSD (N-OPSD) to turn these corrections into supervision at student-visited states. Offline, greedy selection builds a compact pool of frozen experts by rewarding filtered reference-token gains beyond the pool's current best at each position. The highest-peak expert need not provide the best training target. Online routing therefore separates the anchor direction from its level of support. MaxPeak selects the anchor token, and quantile selection chooses among experts whose top token matches it. The student learns from the chosen expert's full next-token distribution through the clipped forward-KL objective inherited from OPSD. We evaluate on AIME 2024, AIME 2025, and HMMT February 2025. Across three independent runs per method, Neighborhood OPSD improves the three-benchmark Average@12 over OPSD by 2.75, 1.67, and 1.94 points on Qwen3-1.7B, 4B, and 8B, respectively. Student-prefix continuations support using the pool beyond the reference trajectories used for selection. Matched ablations support filtered reference-token gains as a selection criterion. Accounting for overlap within the pool and routing by state further improve student accuracy. Inference uses only the distilled student.

Wed 30 SeptComputation and LanguageMachine Learning
The gist
Training AI models that solve math problems can be helped by learning not just from one expert solution, but from a group of slightly different expert views nearby. The authors show that these nearby experts provide useful corrections at different steps of solving a problem, giving better guidance to the model. Their method, called Neighborhood On-Policy Self-Distillation, selects the best experts dynamically and improves model accuracy on math benchmarks compared to previous approaches. This approach learns from the full advice of the selected expert, not just the top suggestion, helping models generalize better.
Open → 2609.39687v1

Confidence measures of large language models struggle to spot their own errors

On the Limits of Metacognitive Monitoring in LLMs

Abstract: Reliable decisions depend on recognizing when an answer may be wrong. In biological cognition, metacognitive monitoring can dissociate from task performance, raising the question of how closely solving and judging are linked in language models. Here we study the confidence reports of four frontier models across 15 benchmarks. High task accuracy can coexist with weak error discrimination: a model solves 97% of competition mathematics problems while its answer-time confidence ranks correct answers above errors barely better than chance. Confidence separates correct answers from errors more effectively on questions solved by a separate reference model, while review brings limited improvement on reference-hard questions. Aggregate discrimination also rewards ranking correct answers on easy questions above errors on hard ones, which question-only forecasts already do well. Cross-evaluation helps most where the evaluator answered correctly, and errors shared by the two models usually retain high confidence. Hard questions and shared errors remain difficult targets for prompted self-review and peer oversight, even in models with strong problem-solving performance.

Mon 28 SeptArtificial Intelligence
The gist
Deciding if an answer is wrong is important for making good choices. The authors checked how well large language models can judge when their own answers might be mistakes. They found that even when these models solve hard problems very accurately, their confidence scores often can't clearly tell a right answer from a wrong one. Trying to review answers or compare with another model helps a little but doesn’t solve the problem for hard questions. So, models are still not very good at knowing when they are wrong.
Open → 2609.34864v1

Hierarchical sampling improves math reasoning in language models

HISPO: Hierarchical Importance-Sampling Policy Optimization with Entropy-Derived Segments

Abstract: Reinforcement learning with verifiable rewards (RLVR) has become a central approach for improving mathematical reasoning in language models, but long-form completions introduce a difficult credit-assignment problem: different parts of a solution trace may contribute unevenly to final correctness. Existing policyoptimization objectives for RLVR commonly apply importance-sampling correction at either the token level (GRPO, DAPO) or the sequence level (GSPO), imposing different granularities for assigning credit across a response. We introduce Hierarchical Importance-Sampling Policy Optimization (HISPO), a segment-level policy-optimization method that constructs rollout-time entropy-derived contiguous segments, assigns soft entropy-based saliency weights, and applies clipped importance-sampling correction at the segment granularity. This provides an intermediate correction unit between token-level GRPO/DAPO and sequence-level GSPO. We evaluate HISPO by fine-tuning Qwen3-1.7B-Base on mathematical reasoning tasks. Across six benchmarks, HISPO improves Pass@8 over the strongest baseline on all benchmarks and matches or exceeds the strongest baseline in Acc@8 on five of them. On AIME25, HISPO improves over GRPO by +3.75 Acc@8 and +3.78 Pass@8, and over GSPO by +2.50 Acc@8 and +1.27 Pass@8. These results suggest that segment-level correction is a promising granularity for RLVR in long-form mathematical reasoning.

Mon 14 SeptArtificial IntelligenceMachine Learning
The gist
Reinforcement learning helps language models improve at solving math problems by learning from rewards, but it's hard to figure out which parts of a long answer actually led to the final correct solution. The authors introduced HISPO, a method that breaks down answers into meaningful segments based on how uncertain the model was when generating each part, then assigns credit to those segments instead of individual words or whole answers. They tested HISPO on several math datasets and found it generally led to better performance than previous methods. This suggests that dividing answers into segments helps models learn more effectively from long math problem solutions.
Open → 2609.15471v1

CircuitLens finds problem difficulty rankings can mislead reinforcement learning

CircuitLens: Reasoning Circuits as Data Selection Signals for Reinforcement Learning with Verifiable Rewards

Abstract: Reinforcement learning with verifiable rewards (RLVR) is sensitive to which problems a model trains on, yet existing selection criteria--difficulty filtering, hand-curation, reward-trajectory scoring--assess data value as an intrinsic property of problems, independent of the model that will learn from them. We introduce Circuit Reasoning Score (CRS), a selection signal derived from 46 reasoning-sensitive attention heads identified via contrastive ablation, computed in a single forward pass on the frozen base model without reward labels or rollouts. CRS runs against the intuitive hypothesis that stronger reasoning-circuit engagement produces better training data: on Qwen2.5-Math-7B, the lowest-engagement decile improves over random selection on three medium-difficulty benchmarks (GSM8K +2.0 pp, OlympiadBench +1.6 pp, Minerva +2.9 pp), while the highest-engagement decile gains less and is indistinguishable from the middle decile. The advantage has boundary conditions: on a domain-curated pool no selection method separates from the others; at 1.5B scale the useful direction differs; and the lowest-reward training condition produces the strongest downstream generalization. Within the Qwen2.5-Math settings tested, RLVR data selection appears regime-dependent rather than reducible to a static ranking of problem quality.

Mon 7 SeptComputation and LanguageMachine Learning
The gist
Choosing which problems to train on is important for reinforcement learning with verifiable rewards (RLVR), but usual methods rank problems on their own merits, ignoring the specific model learning from them. The authors introduce a new way called Circuit Reasoning Score (CRS) that measures how much certain reasoning circuits in the model engage during problem solving. Surprisingly, the easiest problems by this measure sometimes lead to better learning outcomes than the hardest ones. Their experiments show that selecting training data based on reasoning circuit signals depends on the training setup and is not a fixed way to rank problem quality.
Open → 2609.07183v1