AgentLens improves memory control for co-located large language model agents
AgentLens: Framework-Aware Characterization and Resource Control for Heterogeneous LLM Agents
Operating Systems
Summary
Large language model (LLM) agents run on shared computers but their memory and resource needs are not well understood, which makes it hard to manage them efficiently. The authors show that the framework used to run these agents has a big impact on resource use, more so than the specific tasks they perform. They created AgentLens, a system that watches how these frameworks use memory and controls resources smartly without changing the frameworks themselves. This helps keep memory usage stable and reduces delays when multiple agents run together on the same host.
What this means in practice
- •For cloud infrastructure teams: Manage memory allocation for multiple co-located LLM agents to reduce latency and improve resource use without modifying agent frameworks.
- •For ai service operators: Control resource use of diverse LLM agents running on shared hardware to ensure reliable performance under memory constraints.
Authors
Wentao Hong
Abstract
LLM-based agents are increasingly deployed on shared hosts, yet their host-side resource demands remain poorly understood, limiting effective resource coordination. We present AgentLens, which combines a cross-framework workload characterization with a framework-aware resource controller for co-located agents. Across 10 frameworks and 11 matched coding tasks, we find that framework orchestration strongly shapes resource demand: framework identity explains 52.95% of peak RSS variance, compared with 4.22% for task identity, while execve events per tool call differ by 39x between MetaGPT and CAMEL. Resource dimensions are not interchangeable: peak RSS and subprocess count have a Spearman correlation of 0.13. Static budgets and framework-agnostic policies miss these structural differences and the alternation between API waiting and tool execution. AgentLens combines offline framework fingerprints with online execution-phase signals to coordinate memory through cgroup v2, without modifying framework internals. In trace-driven replay across three co-location pairs at a memory budget of 90% of aggregate peak demand, AgentLens keeps the protected worker's P95 allocation latency at 5.7-7.1 ms, improving on static allocation by 2.1-2.6x.