Don't Drop the BATON: Long-Horizon Robot Manipulation via Agentic Subtask Exploration and Transition-aware Memory
2026-08-17 • Robotics
RoboticsArtificial IntelligenceComputer Vision and Pattern Recognition
AI summaryⓘ
The authors address the challenge of chaining many robot skills to complete long tasks, where mistakes can build up and cause failures. They introduce BATON, a method that breaks the problem into smaller parts, explores and solves each part separately, and then combines these solutions to perform the full task more reliably. BATON also uses a system to check if the robot's environment is ready before each skill and to manage transitions between skills smoothly. This approach improves success rates on a difficult long-task benchmark without changing any model parameters. Overall, the authors show a way to better manage complex robot tasks by focusing on smaller steps and smart transitions.
long-horizon robot manipulationvision-language-action (VLA) modelslarge language model (LLM) agentmulti-stage task chainingexploration costtransition-aware memorysubtask explorationrobotic benchmarkstask success rateanalytic primitives
Authors
Bingxin Xu, Yuzhang Shang, Emilio Ferrara
Abstract
Long-horizon robot manipulation chains many contact-rich skills into one multi-stage task. Vision-language-action (VLA) models increasingly master the individual skills, yet the chain still fails: errors compound beyond the policy's ability to correct, and one subtask silently constrains the next. A promising recipe freezes the VLA and puts an LLM agent in charge: it plans in language, moves in free space with analytic primitives, invokes the VLA only for contact-rich segments, and writes adaptation into language memory. Applied to long horizons, it breaks twice. (1) Competence comes from whole-task exploration at test time, whose cost is multiplicative in stages: if one stage needs T episodes, a K-stage task needs about T^K, and a failure does not reveal which stage caused it. (2) It has no representation of transitions: the VLA primitive carries an exit but no entry condition, so a subtask can succeed in a form its successor cannot use. We present BATON. Against (1), BATON makes the subtask the unit of exploration: each is explored in the cheap short-horizon regime and its solution stored in memory; a long-horizon trajectory is then composed from these solutions rather than discovered whole. Cost becomes additive (T*K) and every failure is attributed to a single stage. Against (2), BATON equips exploration with a transition-aware memory. Within a subtask, a verifier agent governs the invocation transition: the VLA is called only after the wrist view confirms the scene is ready. Across subtasks, a handoff transition restores an entry state disturbed by the predecessor's residue, and a lookahead transition selects the strategy whose outcome the successor can inherit. No parameters are updated. On the long-horizon benchmark RoboMemArena, BATON improves task success by 11.6% and cumulative success by 14.9% over the SoTA.