AgentPro reduces AI token use and speeds up task workflows
AgentPRO: Coordinating Compiler Optimizations for LLM Agent Workflows
Software Engineering
Summary
When AI-based agents work on tasks, they often ask the model many times, causing delays and extra effort. The authors present AgentPRO, a tool that smartly combines work steps and cuts down unnecessary AI requests. This tool checks how tasks are planned and updates the work in real-time to save time and resources. Tests showed that AgentPRO used fewer tokens and ran faster compared to existing methods while successfully finishing most tasks.
What this means in practice
- •For ai application developers: Improve efficiency in AI systems by reducing redundant model calls and execution steps using AgentPRO's workflow optimizer.
- •For cloud service operators: Lower costs by cutting AI token usage and speeding up large language model workflows in agent-driven cloud applications.
Authors
Jie Ren, Zheng Wang
Abstract
LLM agents repeatedly return to a model to generate actions and interpret tool results, incurring repeated context processing and generation overhead. Yet a value needed to execute an action need not be observed by the model to generate that action. This distinction creates an optimization opportunity: reconsider where model decisions are necessary, together with the work performed between them. We present AgentPRO, a compiler framework for optimizing agent workflows as execution information becomes available. AgentPRO organizes planned duties, generation regions, and concrete operations in a hierarchical representation, distinguishing execution dependencies from observations required for generation. Region transformations combine compatible duties into fewer model requests, while operation transformations reduce redundant context and tool work. These scopes interact: eliminating a redundant read can remove an observation requirement that otherwise separates generation regions. AgentPRO coordinates transformations using initial plans, current observations, and historical execution profiles through just-in-time, adaptive, and profile-guided optimization. Runtime guards check state-dependent assumptions and retain fallback paths. We implement guarded workflow transformations for AppWorld agents. On 115 tasks selected by successful historical baseline executions, automatic AgentPRO reduces all-attempt mean token consumption by 41.85% and mean component time by 6.61% relative to those baselines, successfully completing 110 of 115 tasks.