Papers for

software developers for chatbots

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Prefix tuning controls outputs of pretrained transformers reliably

Prefix-Tuning for Arbitrary Output Sequences on Pretrained Transformers

Abstract: Prefix-tuning is a parameter-efficient fine-tuning method for large language models (LLMs) that steers model outputs by prepending task-specific prefix representations to the input sequence. Despite its practical success, the theoretical understanding of the expressiveness of prefixes remains incomplete. In particular, the following question remains open: Given a pretrained autoregressive Transformer, for any input-output sequence pair $(\mathtt{x},\mathtt{y})$, does there always exist a prefix $\mathcal{S}$ such that prepending $\mathcal{S}$ to $\mathtt{x}$ induces the model to generate the target output $\mathtt{y}$? In this work, we prove that under the assumption that $\mathtt{y}$ is in the range of the pretrained Transformer, such a prefix $\mathcal{S}$ exists and can be explicitly computed by solving multiple linear systems. We further show that this assumption is necessary, in the sense that such a prefix may not exist when the assumption fails, thereby providing a complete answer to the above question. Our analysis is constructive and the obtained theoretical results hold under fairly general assumptions on model architecture. Extensive experiments on GPT-2 and Qwen3-4B demonstrate that our algorithm produces prefixes with strong control over model outputs.

Fri 16 OctComputation and LanguageArtificial IntelligenceMachine Learning
The gist
This work answers whether it is always possible to steer a large pretrained language model to produce any desired output by adding a special 'prefix' before the input. The authors prove that if a target output can already be generated by the model, then there exists a prefix that makes the model produce that output every time. They also provide a way to compute this prefix explicitly by solving mathematical equations. Their results explain when prefix-tuning works and when it cannot, and they tested this on models like GPT-2.
Open → 2610.24123v1

Llm agents learn better from critiques plus corrected actions

Beyond Corrected Actions: Training LLM Agents on Textual Critiques

Abstract: Large language models (LLMs) are increasingly used for multi-turn agentic tasks that require interactions with users and environments over extended sequences of actions. A common approach to improving LLM agents is to use more capable expert models to provide training supervision, either by behavior cloning the expert's trajectories or by using the expert to correct the actions in trajectories generated by the weaker student model. However, expert LLMs can provide richer supervision than actions alone: they can directly generate textual critique that explain why specific actions are incorrect. We study whether training on such feedback benefits LLM agents and propose Critique-Augmented Agentic Fine-Tuning (CAFT), a framework that elicits textual expert feedback on student-generated trajectories. For each incorrect action, the expert yields both a corrected action and a textual critique explaining the student's mistake--which we then use as the training target for fine-tuning. Across four student models from the Nemotron 3 and Gemma 4 families on the agentic domains $τ^2$-Bench Airline+Retail and WebShop, adding critiques to action-level supervision improves performance by 8% on average over action-only distillation. Additionally, CAFT checkpoints can also provide a better warm start for reinforcement learning than expert-trajectory distillation. Our results demonstrate that textual critiques can strengthen supervision for multi-turn LLM agents.

Wed 14 OctComputation and LanguageArtificial Intelligence
The gist
Large language models (LLMs) that act like agents often make mistakes during multi-step tasks. The authors show that instead of just teaching these models the right action, giving them explanations about why their action was wrong helps them learn better. They created a training method called CAFT that adds these textual critiques alongside corrected actions as feedback. Models trained this way performed about 8% better on several tasks compared to just learning from corrected actions alone. This approach also helps when fine-tuning models before applying reinforcement learning.
Open → 2610.19367v1