Unified model improves robot actions by predicting and grounding transitions
UniMPA: A Unified Memory-Prediction-Action Model via Action-Grounded Transition Modeling
Robotics
Summary
Robots often struggle to figure out what action to take next because similar-looking scenes can lead to very different outcomes, and what seems possible in a prediction might not actually work in the real world. The authors created UniMPA, a system that helps robots better understand how their actions change things by remembering past successful experiences and adjusting plans based on the current situation. This model predicts what future states should look like while checking if those predictions match real, doable actions, making robot behavior more reliable and adaptable.
robotic manipulationvision-language-action modelstransition ambiguityfuture state predictionmemory bankaction groundingphysical executabilitycontext-aware adaptationlatent representationprototype-based refinement
Authors
Wei Li, Rui Shao, Jie He, Lingsen Zhang, Ziwei Liu, Liqiang Nie
Abstract
Recent advances in Vision-Language-Action (VLA) models have improved robotic manipulation, yet observation-to-action learning remains limited by a fundamental transition realizability gap, manifested in three tightly coupled problems: (i) Transition ambiguity. Visually similar current observations may correspond to different manipulation phases and imply different subsequent transitions. (ii) Prediction--execution mismatch. A visually plausible predicted future observation does not necessarily correspond to a physically realizable transition. (iii) Experience--realization mismatch. A historically executable action pattern may not necessarily realize the intended transition in the current scene and therefore requires context-aware adaptation. Accordingly, we propose UniMPA, a Unified Memory-Prediction-Action model that addresses these problems through a shared action-grounded transition interface. (i) UniMPA introduces Persistent-Selective Future Prediction to resolve transition ambiguity by modeling the intended future state evolution. A persistent latent stream continuously tracks task-level progress, while a transition-critical pixel stream selectively resolves fine-grained interaction changes through memory-grounded prediction. (ii) To assess the physical executability of the anticipated transition, the predicted transition queries a temporal Visual-Action Memory Bank. The bank retrieves historically realized visual-action experience, grounding future prediction in executable evidence. (iii) To adapt executable experience to the current scene, an Action-Visual Memory Bank retrieves visually grounded action prototypes from historical action evolution. Prototype-Biased Flow then shifts the flow source toward a historically supported action manifold for context-aware refinement.