Instruction order shapes conflicts in large language model agents
Position, Role or Label? A Controlled Study of Instruction Conflicts in LLM Agents
Computation and LanguageArtificial IntelligenceSoftware Engineering
Summary
Large language model (LLM) agents often receive conflicting instructions from different sources. The authors studied which instructions have more influence by testing order, labels, and roles across many scenarios and models. They found that the first conflicting instruction in one message usually wins, but this effect lessens when conflicts span multiple messages. Models behave differently depending on message type and task, and only a small fraction of responses show awareness of instruction conflicts. This helps understand how LLM agents prioritize instructions in complex settings.
What this means in practice
- •For ai system developers: Design instruction strategies that minimize conflicts by controlling instruction order and message roles to improve LLM agent reliability.
- •For software tool integrators: Improve automation pipelines by understanding how LLM agents resolve conflicting instructions from tools and memory across messages.
Authors
Anish Laddha, Nitesh Pradhan, Gaurav Srivastava
Abstract
LLM agents take instructions from the operator, repository configuration, user, tool results and saved memory, which often conflict. Instruction-hierarchy benchmarks ask whether the higher-ranked source wins, but a source sets position, role and label at once. First, we build WhoWins, 1,080 code-scored scenarios in five task types, each with up to five mutually exclusive instructions, and seven controlled experiments that vary only how instructions are delivered. Second, we study 14 open-weight models (0.6B to 122B, thinking off and on) over 1,887,356 scored replies, checking main findings on held-out scenarios. Inside one message the first of two conflicting lines wins 72.42 to 94.66%, and across messages it keeps 22.95 to 62.65%. A policy sent first in the system (or developer) message loses most conflicts with later lines in every Qwen model but wins most on gpt-oss when it acts. In tool and file-write tasks it wins less often than the same text sent first as a labelled user message, in every model-mode. Third, we test whether pairs forecast larger conflicts; forecasts improve in every model-mode once they know where the task line is. Finally, by our keyword patterns at most 2.92% of decided pair replies flag or hedge a conflict.