Language models struggle more when combining multiple tasks in one prompt
On the Fragility of Tasks under Composition: Implicit Budgeting in Language Models
Computation and LanguageArtificial IntelligenceMachine Learning
Summary
Language models can handle tasks well when given separately but make more mistakes when asked to do several tasks together in one prompt. This problem is not just because errors add up; instead, models seem to share a hidden token 'budget' that they spend less per task during combined tasks. Tasks needing longer reasoning are especially fragile and suffer more errors when combined. The authors show that by grouping tasks based on this fragility, it's possible to reduce token use significantly without losing accuracy.
What this means in practice
- •For natural language engineers: Optimize multi-task prompts by grouping tasks to reduce token use while maintaining response accuracy.
- •For ai system integrators: Design systems that allocate implicit token budgets efficiently when combining multiple tasks in language model pipelines.
- •For automated customer support teams: Improve chatbot performance on multi-part queries by managing how tasks are composed to limit mistakes.$Commercial implications: Enables chatbot product enhancements in handling complex multi-question interactions with fewer errors.
Authors
Chenpan Li, Thies Brockmoeller, Stefanie Jegelka, Sharut Gupta, Nikolaos Karalias
Abstract
Modern language models are increasingly asked to handle several tasks within a single prompt. Doing so requires solving each task correctly while integrating the resulting answers into a coherent response. In this paper, we identify a fundamental limitation in this process, where models make substantially more errors when tasks are composed, even when they can solve every constituent task reliably in isolation. This degradation cannot be fully explained by the accumulation of independent errors. Across both open-weight models and the frontier model Claude Opus 5, our findings instead point to an implicit thinking budget underlying this effect. In particular, under composition, models use substantially fewer tokens per task than in isolation, even when the full sequence remains well within the context window. Tasks that require longer solution traces in isolation are consequently more $\textit{fragile}$ under composition, i.e., models make more errors on prompts containing these tasks. In isolation, non-fragile tasks receive more reasoning than they need and waste compute. However, by grouping tasks according to their fragility, the implicit budgeting can be manipulated to reduce token usage by $43$\% across a range of math and linguistic tasks while also preserving accuracy.