Robot control improves with explicit memory of past concepts
ECoMEM: Explicit Concept Memory for Memory-Dependent Robot Control
Robotics
Summary
Robots often need to remember objects, tasks, or demonstrations they saw earlier, but they forget when that information is no longer visible. The authors separate what a robot should remember from how it acts, by creating a clear memory system that stores important past facts as concepts grounded in what the robot senses. This makes the robot better at long tasks because it keeps track of what it learned or completed. Their tests show much higher success when the robot uses this explicit memory rather than relying only on current observations.
What this means in practice
- •For robotics engineers: Design robot controllers that store and update specific task-relevant facts during operation for improved long-term task success.
- •For automation system integrators: Integrate explicit memory modules that reuse and adapt conceptual memories across different robot tasks, reducing retraining effort.
- •For assistive robotics developers: Build assistive robots that remember user demonstrations and task progress with an interpretable concept memory for more reliable interaction.$Commercial implications: This enables assistive devices that remember complex user instructions and adapt over time, a competitive advantage in healthcare robotics.
Authors
Yize Liu, Ke Wang, Mac Schwager, Yiqing Xu, Jiajun Wu
Abstract
A robot may lose sight of an object it must later retrieve, need to recall what a person demonstrated earlier, or track which steps of a task it has already completed. Current vision-language-action (VLA) policies often fail once the information needed for action disappears from the current observation, making memory critical for long-horizon robot behavior. Existing approaches typically provide longer histories or learn implicit memory from observation-action trajectories. But action supervision tells a policy how to act, not what to remember: it does not specify which past facts should persist or how they should change as new evidence arrives. We therefore separate maintaining an evidence-grounded account of the past from learning how to act on it. This insight motivates Explicit Concept Memory (ECoMEM), which represents task-relevant history with a reusable library of grounded concepts. An evidence-based Writer selects and updates these records, while a learned Reader turns them into memory tokens that directly condition the VLA. Across 16 RoboMME tasks, ECoMEM leads the evaluated robot policies on 15 tasks. On two new real-robot tasks, the same memory library either transfers directly or requires only one new concept, achieving 86.1% success versus 8.6% for a no-memory VLA. These results show that explicit concepts provide a reusable and extensible memory interface for robot control. Project website: https://ecomem.github.io/