Improving coding agents with better instructions and control sharing
Improving Environment-Agnostic Coding Agents: The Role of Scaffolding
Artificial Intelligence
Summary
Some computer programs called language models can write code to solve problems, but they still do better when given step-by-step guidance from humans. The authors studied how making small changes to an agent’s instructions helped it solve puzzles more effectively, even without environment-specific hacks. They found that simply encouraging the agent to report problems and letting an external controller manage task resets improved results, though it still didn’t match specially tailored versions. This shows that having tools and information is not enough—how the agent uses them matters a lot, and human-designed procedures still help.
What this means in practice
- •For software engineers: Build general-purpose coding assistants that improve problem-solving without environment-specific customization by optimizing agent instructions and control delegation.
- •For automation engineers: Develop automated agents capable of adapting instructions for various tasks to increase their efficiency in unfamiliar environments without extra engineering.
Authors
Sergey Rodionov
Abstract
Stronger language models may increasingly construct useful procedures for themselves, reducing the advantage of developer-supplied scaffolding. We investigate why this advantage persists and whether an environment-agnostic agent can be improved without environment-specific engineering. We evaluate a Codex-based agent in two nearly identical prompt variants on ARC-AGI-3, AGI Maze, and Terminal-Bench. Each variant is deployed with unchanged code, prompts, and general-purpose tools across environments, receiving environment-specific information as input. Changing only an instruction to encourage reporting difficulties and returning control for automatic continuation improves mean scores on both ARC-AGI-3 and AGI Maze, while Terminal-Bench shows small, mixed changes. Yet the general agent remains below an ARC-AGI-3-specific reference, including with GPT-5.6-sol. The reference prescribes organization and control rather than supplying new game rules or solutions. To investigate this gap, we transfer reset decisions to a scripted controller that follows a convention already given to the agent. This small change recovers part of the specialized agent's advantage. A complementary AGI Maze experiment shows that additional tutorial experience can reduce performance. Together, these findings highlight the distinction between having information and tools available and using them effectively. Supplied procedures still matter for the models tested here. The performance gains also come with higher token-based cost. The experiments do not separate better use of information and procedures from additional or differently allocated inference. We provide the EnvTraverse framework and an evaluated baseline agent for testing whether further environment-independent changes improve performance in some environments without degrading it in others.