EmbodiedMemory-Bench tests memory for long tasks in robots and agents
EmbodiedMemory-Bench: Benchmarking Embodied Memory for Long-Horizon Embodied Tasks
Computer Vision and Pattern Recognition
Summary
Robots and virtual agents that act over long periods need to remember details about their surroundings to do well. The authors found current agents often forget or mix up what they saw or did, leading to mistakes. They created a new test called EmbodiedMemory-Bench to check how well agents remember and update facts during complex tasks. They also built a new memory system, Embodied-Memorizer, that organizes information better, and showed it helps agents perform stronger on this test. Still, many memory challenges remain for current AI models.
What this means in practice
- •For robotics developers: Evaluate and improve robot memory systems in simulation for tasks requiring remembering and updating environment details over time.
- •For virtual assistant engineers: Test and advance virtual agents' abilities to maintain and use long-term memory for multi-step interactions with changing environments.
Authors
Lizhou Liang, Xinyu Zhong, Miao Pan, Xiaohe Zhou, Xuanyu Liu, Qinfeng Li, Peng Li, Jintao Chen, Xuhong Zhang, Wenqi Zhang
Abstract
Long-horizon embodied interaction requires agents to retain and continually update information about the environment as they observe, act, and encounter change. Yet current agents struggle to maintain such memory reliably. Our analysis traces this limitation to four key deficiencies: weak fine-grained visual memory, unreliable dynamic world-state tracking, failing to record world state revealed by interaction outcomes, and limited generalization from prior experience. However, existing benchmarks do not directly assess these memory capabilities during long-horizon embodied interaction. To address this gap, we introduce EmbodiedMemory-Bench (EMem-Bench), comprising 2,554 interactive episodes across four task families. EMem-Bench requires agents to build and update memory from interaction history, then use it to complete a later task by acting in the environment. We further present Embodied-Memorizer (EMem), an external memory system that organizes embodied experience into spatial, event, and scene memories. We also train EMem-8B, an 8B policy that manages and uses these memories. We evaluate a diverse range of open-source and proprietary MLLMs and representative multimodal memory systems. Results show that current models remain weak and uneven across the four challenges. Under matched backbones, EMem achieves the best overall performance among the evaluated memory systems and improves both open-source and proprietary models, while EMem-8B further improves over its backbone. Project page: https://zju-omniai.github.io/EmbodiedMemoryBench/