Robot memory flaws cause false progress signals in counting tasks
Is Your Robot's Memory Just a Clock? Temporal Shortcuts in Memory-Augmented VLA Policies
Robotics
Summary
Robots that use memory to remember what happened may mistakenly think they made progress just because time passed, even if they were idle. The authors show that when a robot pauses briefly, its current memory system treats the pause as work done and stops counting early, hurting task success. This problem happens because the robot's memory relies on tracking time rather than actual events. Changing memory to count real events instead of clock ticks helps fix the issue without retraining the robot.
What this means in practice
- •For robotics developers: Improve robot task memory by indexing events rather than time to avoid false progress signals during pauses.
- •For industrial automation teams: Enhance automated systems’ reliability by preventing premature task completion caused by clock-based memory errors.
Authors
Chubin Zhang, Bo An
Abstract
Memory-augmented vision-language-action (VLA) policies are meant to remember what happened. Many of them, however, fill memory by time: they sample frames uniformly across the episode and stamp each frame with its time step. Because time passes even when nothing happens, such a memory may not tell waiting from working. We show that it does not. On the counting tasks of RoboMME, we make the robot briefly pause or fidget its wrist. Such a null event changes nothing the task depends on, yet a released memory VLA counts the interval as work done and stops early. We call this failure phantom progress. It cuts success from 47 to 16 of 50 on one task and from 44 to 10 on the other. When the same frames are withheld from memory, success returns, so the cause lies in memory rather than in the interruption itself. We trace the failure to a temporal shortcut: because demonstrations proceed at a steady pace, elapsed time predicts progress. Indeed, rescaling the time stamps alone makes the policy stop early when the clock runs fast and repeat when it runs slow. Surprisingly, removing the time encoding does not cure the failure, because the frames of a null event still occupy slots of the time-sampled memory. Across RoboMME's released designs, vulnerability follows how memory is indexed. Of the fixes the diagnosis suggests, training with injected pauses does not remove the failure and impairs counting, whereas indexing memory by events recovers half or more of the loss without retraining. A memory meant to remember what happened should count events rather than clock ticks.