Versioned execution improves task updates for interactive ai agents
Serving a Revisable World: Versioned Execution for Interruptible Agents
Distributed, Parallel, and Cluster Computing
Summary
Sometimes AI agents that work on tasks need to change what they’re doing when the user updates instructions or something goes wrong. Usually, the old work is stopped and restarted from scratch, which can waste time and cause confusion. The authors created a method called versioned execution that lets the system stop outdated work quickly while reusing valid progress made before. This approach helps the next attempt start faster and avoids releasing wrong results. They tested it in a system running AI models, showing it makes updates smoother and quicker.
What this means in practice
- •For ai system engineers: Improve interactive AI agents by enabling fast, coordinated task updates that reuse previous valid results instead of restarting from scratch.
- •For cloud platform operators: Enhance multi-tenant AI services by reducing obsolete output and accelerating task revision times in distributed model serving.
Authors
Yanxin Zhang, Rahul Sharma, Nitin Vegesna, Zheyu Fu, Chang Liu, Trivikram Krishnamurthy
Abstract
LLM agents revise running tasks when users change instructions, tools fail, or new information changes a plan. Today's servers express a revision as aborting old requests and submitting replacements. Yet the old execution's buffered output and outstanding work must stop affecting the application, while completed KV state may still be useful to its replacement. Handling these obligations separately can leave obsolete effects publishable and force the successor to rebuild valid state. We present \retire, a serving control-plane redesign around \emph{versioned execution}. Requests own scheduling and memory resources; execution versions own \emph{authority}, the permission to publish output or install state for the current execution. \retire first revokes obsolete work, then bounds its remaining execution and certifies the completed prefix its successor can inherit. The successor runs from that state while isolated old resources are reclaimed asynchronously. This unifies fast invalidation and selective preservation in one version transition. We implement \retire in vLLM across output publication, GPU execution, KV handoff, tiered recovery, and distributed and multi-tenant serving. Correctness experiments verify current-version output and valid state inheritance across these paths. Combining invalidation with inheritance reduces revision-to-successor time-to-first-token by a median 17.1\% in controlled paired experiments. A replay of recorded coding-agent interruption arrivals emits no obsolete output and keeps every final version progressing through repeated revisions. \retire turns abort-and-restart into a coordinated handoff that stops obsolete work quickly and preserves useful work for its successor.