Retriever: Composing Closed-Loop Asynchronous Robot Programs
Robotics
Summary
The authors address the challenge of building robot systems that need to handle tasks running at different speeds and delays by creating Retriever, a complete framework that manages timing explicitly. Retriever models robot decision-making as connected functions working on streams of data with clear timing rules. This approach helps avoid unpredictable behavior caused by hidden timing issues, making robot programs easier to debug and reuse. They tested Retriever on a real robot and measured its performance and ability to replay past actions exactly.
Authors
Linfeng Zhao, Haojie Huang, Jiayuan Mao, Weiyu Liu, Mykel Kochenderfer, Lawson L. S. Wong
Abstract
Building long-horizon robot agents requires composing closed-loop pipelines -- perception, belief update, planning, and control -- whose components run at different clocks and with variable latency. Today, these systems are often assembled with ad-hoc concurrency and pub/sub conventions that make timing and input-consumption semantics implicit, yielding schedule-dependent behavior that is hard to reproduce, debug, and reuse. Current solutions typically solve parts of this problem at either the algorithmic or the systems layer, but not both. In this work, we propose Retriever, which spans the entire stack: an asynchronous decision model, a programming model, a runtime, and an example closed-loop agent pipeline. Retriever represents an agent as a graph of stateful causal stream functions executed on explicit run clocks. We formalize this view via an asynchronous environment-agent loop over continuous-time streams and show that finite-memory causal policies can be represented by compositions of these operators. Retriever compiles these graphs into a runtime that supports multiple backends, enabling systematic debugging across running environments and deterministic replay from logged asynchronous data. We evaluate Retriever through a real-robot case study together with controlled studies of runtime overhead and deterministic replay behavior.