Robot control improves by planning subtasks before acting
Plan First, Act Later: Streaming Reasoning for Executable Memory in Robot Control
Robotics
Summary
Robots working on long and complicated tasks need to remember important details over time. The authors created a system called PAL that plans out what the robot should do step-by-step before actually doing it. This system keeps track of a short visual memory, events it has seen, and a list of subtasks to do. This approach helps the robot work better by focusing on the right subtask at the right time, leading to higher success rates in tests with both computer simulations and real robots.
What this means in practice
- •For robotics engineers: Improve robot manipulation in tasks needing memory by integrating proactive subtask planning to boost control success rates.
- •For automation system designers: Create hierarchical memory architectures for robots that better handle long sequences of actions in real-world environments.
Authors
Qinghang Liu, Conglin Wang, Zongsheng Li, Boyong He, Han Gao, Pengcheng Wang, Tao Sun, Lige Liu, Chang Liu
Abstract
Long-horizon robot manipulation requires a robot to remember task-relevant information beyond its short observation window. Existing memory-augmented policies typically retrieve relevant information from history and reason over it at execution time, placing a heavy reasoning burden on the high-level model. To address this challenge, we propose PAL (\textbf{P}lan first, \textbf{A}ct \textbf{L}ater), a hierarchical memory framework for vision-language robot manipulation. It maintains a short-term visual buffer, a semantic-event memory, and an executable \texttt{subtask\_list}. A high-level vision-language model (VLM) converts streaming visual observations into the \texttt{subtask\_list}. At execution time, the VLM selects the currently relevant subtask from the \texttt{subtask\_list} and passes it to a low-level vision-language-action (VLA) policy. To teach the model what to write to memory and when to update it, we introduce a two-stage data generation procedure that first expands sparse update supervision and then uses self-labeling with task-grounded verification to generate training data that better match the state distribution encountered during execution. We evaluate our approach on memory-dependent benchmarks and real-world manipulation tasks. On RoboMME, PAL achieves a \textbf{56.05\%} success rate, outperforming the strongest baseline, FrameSamp-Modul, at 44.51\%. In real-world experiments across four manipulation tasks, \pal{} reaches \textbf{60.83\%} overall success, surpassing MemER (45.00\%) and FrameSamp-Modul (28.33\%). These results show that proactively writing future-relevant subtasks into hierarchical memory improves long-horizon visual reasoning and robot control. Code and videos can be found on our website: https://qinghangliu.github.io/pal-webpage/