Agentic robot fleets need careful workload and resource management

Serving Agentic Robot Fleets: Workload Characterization, KV-Cache Thrashing, and Admission Control

RoboticsDistributed, Parallel, and Cluster Computing

Summary

Agentic robots use a complex setup combining language understanding and actions, but running many such robots at once is challenging because of two main resource limits. One is the large memory needed to remember past interactions (key-value cache), and the other is the tight timing needed to perform actions quickly. The authors studied how these robots perform on specific tasks using powerful GPUs and identified how workload and concurrency affect performance. They also suggest practical ways to manage how many robots run simultaneously to keep the system efficient.

What this means in practice

  • For robotics system engineers: Optimize resource allocation and concurrency limits when deploying fleets of agentic robots to maintain performance under strict timing constraints.
  • For cloud service operators: Manage GPU server workloads for serving multiple AI-powered robots by balancing memory use and action execution to improve system throughput.

Authors

Sida He

Abstract

Agentic robots combine a vision-language brain, a tool-using harness, and a vision-language-action (VLA) policy. Serving a fleet of these agents introduces two coupled resource constraints: the brain repeatedly revisits long, growing key-value (KV) cache states, while the VLA executes action chunks under tight control deadlines. We characterize 80 LIBERO-PRO episodes generated by GPT-6-Astra, RPent, and pi0.5 (1,862 brain turns and 1,582 VLA inference chunks), profile both layers on H800 graphics processing units (GPUs), and replay the resulting agent programs on a vLLM serving engine. Brain contexts reach 215k tokens; the median Astra request takes 7.78 s, while the intervening environment-step gap is only 1.0 s. Under a 0.25 s action-chunk deadline, one VLA GPU meets the 1% starvation target for one robot; under a 1.0 s deadline, its measured capacity reaches at least 32. In sustained brain replay, increasing concurrency from 20 to 32 robots reduces throughput from about 2.4 to 0.6 requests/s as repetitive prefill displaces generation. A static active-episode cap restores 2.24-2.25 requests/s and raises completed trace-program throughput from 1.30 to 5.00-5.55 episodes/min. Our measurements expose the workload-dependent shift between action deadlines and brain KV residency, and establish practical principles for capacity planning and admission in sustained agentic robot fleets.