Papers for

software developers integrating ai assistants

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Fine-tune large language models efficiently using noisy predictions

Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching

Abstract: Collecting large-scale, diverse, and high-quality post-training datasets for autoregressive language models is challenging. Consequently, Supervised Fine-Tuning (SFT) is typically applied to datasets significantly smaller than those used for pre-training, which leads to overfitting and the catastrophic forgetting of the tails of the data distribution. To prevent this collapse, we introduce Ambient SFT, a principled framework for leveraging imperfect predictions to augment the fine-tuning set. In our method, a "frozen" version of the pre-trained model serves as a noisy teacher that the fine-tuned model must match at a coarse level. In particular, we require that the high-temperature versions of the conditional distributions produced by the post-training model should match the high-temperature versions of the pre-training ones. The temperature hyperparameter controls the level of trust in the pre-trained model, with the limit $T\to \infty$ reverting back to regular SFT and $T=1$ to knowledge distillation. We demonstrate that our approach outperforms standard SFT across domain adaptation, instruction-tuning, and reasoning, leading for up to $21\%$ average relative improvement to standard SFT.

Tue 17 NovMachine LearningArtificial IntelligenceComputation and Language
The gist
It can be hard to train large language models with small amounts of new data because they forget what they learned before or overfit the new data. The authors introduce a way to fine-tune these models more carefully by having the updated model try to imitate a frozen version of itself, but in a softer, less exact way. This method uses a 'temperature' setting to adjust how closely the new model follows the old one’s predictions. Their approach improves performance in adapting models to new tasks and instructions while avoiding forgetting.
Open → 2611.23229v1

Improving language model reasoning by learning from self-generated answers

Learning from Self-Generated Responses: Reliability--Uncertainty Self-Distillation for Reasoning

Abstract: Reference solutions and question-specific rubrics are not always available to guide the teacher in self-distillation. A large language model's own solution attempts offer an alternative, but their quality is difficult to assess in open-ended tasks without automatic correctness checks, and even correct attempts can involve uncertain token predictions. We propose Reliability--Uncertainty Self-Distillation (\method{}) to learn from these attempts. The same model serves as teacher and student: the teacher receives the question and completed attempt, and the student matches the teacher's token-level predictions using a question-only prompt. \method{} weights each attempt's distillation loss by additively combining judged reliability, estimated from a self-judge's score and confidence, with student predictive uncertainty. Across eleven reasoning benchmarks, \method{} outperforms all five baselines in average accuracy, reaching 70.47\% on Qwen3-4B and 76.69\% on Qwen3-8B (8.90 and 2.10 percentage points above the base models). Data and code are available at https://github.com/JP-25/RUSD.

Tue 13 OctArtificial Intelligence
The gist
It can be hard for AI to learn from its own answers when there’s no clear way to check if those answers are right. The authors came up with a method where the AI judges how reliable its own answers are, and also considers how uncertain it is about parts of those answers. This way, the AI teaches itself better by focusing more on trustworthy and confident answers. Their approach worked better than other methods across many reasoning tests.
Open → 2610.19176v1