Unified model improves robot actions by linking memory prediction and execution
UniMPA: A Unified Memory-Prediction-Action Model via Action-Grounded Transition Modeling
Robotics
Summary
Robots often struggle to decide what action to take when what they see can mean different things, or when their planned actions don’t actually work in the real world. The authors introduce a new model called UniMPA that helps robots keep track of what they’re trying to do, predict what will happen next, and check past actions to make sure their plans can really be done. By combining memory of past experiences with predictions and current observations, the robot can better adapt its actions to the current situation. This approach aims to make robot manipulation more reliable and context-aware.
What this means in practice
- •For robotics engineers: Improve robotic arm control by integrating past action outcomes to verify and adapt planned manipulation steps.
- •For industrial automation teams: Design assembly line robots that predict and confirm feasible actions before execution to reduce errors and downtime.
Authors
Wei Li, Rui Shao, Jie He, Lingsen Zhang, Ziwei Liu, Liqiang Nie
Abstract
Recent advances in Vision-Language-Action (VLA) models have improved robotic manipulation, yet observation-to-action learning remains limited by a fundamental transition realizability gap, manifested in three tightly coupled problems: (i) Transition ambiguity. Visually similar current observations may correspond to different manipulation phases and imply different subsequent transitions. (ii) Prediction--execution mismatch. A visually plausible predicted future observation does not necessarily correspond to a physically realizable transition. (iii) Experience--realization mismatch. A historically executable action pattern may not necessarily realize the intended transition in the current scene and therefore requires context-aware adaptation. Accordingly, we propose UniMPA, a Unified Memory-Prediction-Action model that addresses these problems through a shared action-grounded transition interface. (i) UniMPA introduces Persistent-Selective Future Prediction to resolve transition ambiguity by modeling the intended future state evolution. A persistent latent stream continuously tracks task-level progress, while a transition-critical pixel stream selectively resolves fine-grained interaction changes through memory-grounded prediction. (ii) To assess the physical executability of the anticipated transition, the predicted transition queries a temporal Visual-Action Memory Bank. The bank retrieves historically realized visual-action experience, grounding future prediction in executable evidence. (iii) To adapt executable experience to the current scene, an Action-Visual Memory Bank retrieves visually grounded action prototypes from historical action evolution. Prototype-Biased Flow then shifts the flow source toward a historically supported action manifold for context-aware refinement.