AI inference success improves with informed planning and more tokens
The Organization of Inference: Information, Resource Constraints, and AI Production
Artificial Intelligence
Summary
The paper shows that how AI systems organize their work and share information affects their ability to solve problems. When AI systems plan their steps with access to detailed task information and enough capacity, they succeed much more often than when just trying to solve tasks directly. The researchers tested this on programming tasks, finding that more tokens (a measure of system capacity) and task knowledge significantly boost success rates. They also found the balance between planning and execution changes with resource limits.
What this means in practice
- •For software engineering teams: Improve automated code generation workflows by allocating resources to informed planning stages for higher success in complex tasks.
- •For ai system designers: Design AI workflows that balance planning and execution phases, optimizing capacity and information flow to enhance task performance.
Authors
Yukun Zhang, Kemu Xu, Yishen Chen
Abstract
The economic value of inference depends on how capacity and task information are distributed across stages of AI production. We study these organizational margins using controlled workflow experiments on externally verified software-engineering tasks. In two matched resource panels, direct execution records the same success rate of 59.6 percent at logical-token ceilings of 12,000 and 24,000, while success under information-constrained planning rises from 36.2 to 51.2 percent. The planning disadvantage narrows by 15.0 percentage points (95 percent task-cluster bootstrap interval: 4.2 to 25.8). A strict read-only planning campaign varies whether the planner sees the task issue. At 12,000 tokens, issue access raises success by about 16 percentage points over issue-hidden planning. Compared with direct execution, task-informed planning is about 10 points lower at 12,000 tokens; at 24,000 tokens, it shows a 29.6-point advantage. In the resource panels, direct execution uses substantially less than either ceiling, while the planning workflow's binding rate falls from 46.2 to 0.8 percent and downstream execution accounts for 89.9 percent of the increase in total use. Scale determines the capacity available to a system; workflow and information structure shape the productive value