Skills in Weights, Memory in Code: Hybrid Learning for Memory-Dependent Robot Manipulation
2026-08-10 • Robotics
Robotics
AI summaryⓘ
The authors propose HyMeS, a new approach to help robots perform tasks that require remembering and reasoning over long periods, which is challenging for typical methods that only look at recent observations. Their system combines learned basic motor skills with a higher-level agent that manages memory and decision-making using feedback to improve over time. They also use signals from the robot's body and visual judgments to verify progress and update memory. This approach improves task success and data efficiency compared to previous methods that try to learn everything at once.
vision-language-action policiesMarkoviannon-Markovianimitation learningcoding agentsheuristic learningmultimodal verificationproprioceptioncompositional generalizationrobot manipulation
Authors
Yunhao Zhao, Zhenyang Ni, Haoyang Chen, Ruohan Zhang, Qi Zhu
Abstract
Modern vision-language-action (VLA) policies have acquired broad manipulation skills, but typically generate each action chunk from the current observation or a short fixed-length history. However, real-world manipulation is often non-Markovian, requiring robots to retain and reason over task-relevant information from long-horizon interaction histories to determine the next action. To address this challenge, we propose HyMeS, a hybrid learning framework that leverages the reasoning and memory-management capabilities of coding agents to steer a Markovian VLA for memory-dependent manipulation. Specifically, HyMeS learns low-level motor skills through gradient-based imitation learning, while a coding agent acquires high-level memory-management strategies through heuristic learning by iteratively updating an executable heuristic system from rollout feedback. Furthermore, we close the loop between steering and execution through multimodal stage-completion verification, which updates memory using proprioceptive signals and multi-frame VLM judgments. Compared with end-to-end memory-augmented VLAs, HyMeS requires demonstrations only for reusable motor skills rather than for every history-dependent task configuration, enabling data-efficient compositional generalization. On RoboMemArena, HyMeS improves mean cumulative success from 52.5% to 66.2% and mean task success from 41.3% to 60.1% over pi0.5, while outperforming PrediMem by 4.5 points in cumulative success and 14.5 points in task success.