Many-Tier Instruction Hierarchy in LLM Agents
2026-04-10 • Computation and Language
Computation and LanguageArtificial Intelligence
AI summaryⓘ
The authors explain that language models often get instructions from many sources, each with different levels of importance. Normally, models follow a simple hierarchy with a few clear privilege levels, like system instructions over user ones. But real-world situations are more complicated, with many instructions conflicting at different levels. They created a new system called Many-Tier Instruction Hierarchy (ManyIH) and a benchmark with 12 levels of instruction conflicts to test how well models handle this complexity. Their tests show that current models struggle a lot, highlighting the need for better ways to handle many conflicting instructions safely and effectively.
Large language modelsInstruction hierarchyPrivilege levelsInstruction conflict resolutionAgentic settingsMany-Tier Instruction HierarchyBenchmarkNatural language processingModel evaluationTask complexity
Authors
Jingyu Zhang, Tianjian Li, William Jurayj, Hongyuan Zhan, Benjamin Van Durme, Daniel Khashabi
Abstract
Large language model agents receive instructions from many sources-system messages, user prompts, tool outputs, and more-each carrying different levels of trust and authority. When these instructions conflict, models must reliably follow the highest-privilege instruction to remain safe and effective. The dominant paradigm, instruction hierarchy (IH), assumes a fixed, small set of privilege levels (typically fewer than five) defined by rigid role labels (e.g., system > user). This is inadequate for real-world agentic settings, where conflicts can arise across far more sources and contexts. In this work, we propose Many-Tier Instruction Hierarchy (ManyIH), a paradigm for resolving instruction conflicts among instructions with arbitrarily many privilege levels. We introduce ManyIH-Bench, the first benchmark for ManyIH. ManyIH-Bench requires models to navigate up to 12 levels of conflicting instructions with varying privileges, comprising 853 agentic tasks (427 coding and 426 instruction-following). ManyIH-Bench composes constraints developed by LLMs and verified by humans to create realistic and difficult test cases spanning 46 real-world agents. Our experiments show that even the current frontier models perform poorly (~40% accuracy) when instruction conflict scales. This work underscores the urgent need for methods that explicitly target fine-grained, scalable instruction conflict resolution in agentic settings.