MemSidecar improves agent memory scheduling for long tasks

Who Schedules Agent Memory? A Sidecar Approach to Long-Horizon Tasks

Artificial Intelligence

Summary

Big language model agents often need to remember information to handle long and complex tasks. The authors noticed that the way these agents schedule memory retrieval slows them down and adds to their workload. They created MemSidecar, which separates memory scheduling from task execution using two modes: Hooks delivers needed memory right when it's needed, and Foresight predicts memory needs ahead of time to fetch data early. These approaches help the agents work faster and more efficiently on long tasks.

What this means in practice

  • For software engineers: Integrate MemSidecar to reduce delays in AI systems running extended workflows by managing memory retrieval independently from task execution.
  • For cloud platform teams: Optimize large-scale AI service performance by implementing parallel memory prefetching to lower execution cost and latency in long-running agents.

Authors

Qijia Zhuang, Rui Liu, Yuxiang Ren

Abstract

As large language model agents increasingly undertake complex, long-horizon tasks, memory has become an important component for supporting sustained decision-making and action. Existing methods primarily focus on memory storage, updating, and retrieval, but often overlook how memory scheduling affects task execution: the main agent must identify memory needs, initiate retrieval, and wait for results while advancing the task, increasing both scheduling burden and execution time. To address this problem, we propose MemSidecar, which decouples memory scheduling from task execution through two independently selectable modes. Hooks directly delivers bound memory to the main agent when execution events match predefined conditions. Foresight uses an independent agent to predict potential queries, retrieve results in parallel, and store them in a cache, delivering matching and valid results when actual requests arrive. Hooks and Foresight achieve scores of 41.23% and 44.34% on MCP-Persona, respectively, while effectively reducing execution costs and latency and improving the efficiency of long-horizon agent execution.