Instruction stacks shape and complicate coding agents’ outputs
Hidden in Plain Sight: an Empirical Study about The Instruction Stacks of Coding Agents
Software Engineering
Summary
Coding agents don’t just respond to the prompt you give them; they also use extra hidden instructions that guide how they write code. The authors studied these hidden layers—called instruction stacks—across many versions of different coding agents. They found that these instructions change over time and often include tool definitions and style rules, which can affect the code the agents produce. Because these layers are not clearly documented, it’s hard for users and researchers to fully understand how the code is generated or to track changes accurately.
What this means in practice
- •For software development teams: Improve code review and debugging by considering hidden instruction stacks that influence coding agents’ outputs beyond visible prompts.
- •For tool developers for ai coding assistants: Design better interfaces and documentation tools by capturing and exposing instruction stack changes alongside coding agent releases.
Authors
Felipe de Sant'Anna Paixão, Dhyego Tavares, Eduardo Almeida, Iftekhar Ahmed
Abstract
Coding agents send their models more than the user's prompt: they add instructions, tool definitions, and environmental context that form an instruction stack. This stack guides execution and changes across releases, yet remains largely unexposed to users and insufficiently characterized in research. We present a longitudinal, cross-agent study of the instruction stacks captured in the first model requests of 1,886 releases of Claude Code, Codex, Gemini CLI, Qwen Code, and OpenCode. Under controlled conditions, we analyze their composition and evolution and trace changes in the four open-source agents to documented motivations. Our findings show why examining the system prompt or reporting the model alone provides an incomplete account of agent inputs. Tool definitions dominate three agents' latest requests, while occasional concentrated revisions alter instructions governing tool use, attribution, and coding style. Such revisions can complicate how researchers identify agent-authored contributions and interpret generated code. We also found that only 48% of examined change units directly explain the motivation for stack revisions. Across agents, shared capabilities emerge despite increasingly distinct wording. These findings expose an evolving, incompletely documented layer of agent design. The findings motivate documenting captured instruction stacks alongside agent releases, models, and user prompts, and preserving the rationale behind their revisions.