MemLife improves memory retrieval from long egocentric videos
MemLife: Curating and Reasoning over Long-Term Egocentric Video Memories
Computer Vision and Pattern RecognitionArtificial IntelligenceComputation and Language
Summary
Keeping track of everything in long personal videos is hard because searching through hours of footage takes a lot of time and computer power. The authors present MemLife, a system that turns these videos into short text stories about important people or things, which makes finding answers faster and easier. They also introduce MemOpt, a method that teaches MemLife to write better and more useful memories. These improvements help AI assistants remember and reason about daily life more effectively without needing to watch all the videos again each time.
What this means in practice
- •For personal assistant developers: Build AI assistants that answer questions about months of user videos without reprocessing all the raw footage each time.$Commercial implications: Enables selling advanced AI-based personal video assistants that summarize and recall daily life efficiently.
- •For lifelogging device designers: Create devices that record and automatically generate compact, searchable memories from long-term first-person video streams.
Authors
Guangzhi Xiong, Xinyuan Zhang, Xiao Yang, Hyokun Yun, Kai Zhang, Shiun-Zu Kuo, Hyeonjeong Ha, Xilun Chen, Kai Sun, Lucas Liang, Guangqiang Dong, Ejaz Ahmed, Ahmed A Aly, Anuj Kumar, Raffay Hamid, Aidong Zhang, Xin Luna Dong
Abstract
Long-term egocentric video enables personalized AI assistants to reason about daily life. However, as video histories grow to hundreds of hours spanning months or years, reprocessing raw clips for every query becomes computationally prohibitive. Memory systems offer a scalable alternative by compacting videos into text representations, but often fail on practical benchmarks: either the memory does not preserve key evidence, or the retriever fails to locate relevant entries due to retrieval competition in growing search spaces. To address these challenges, we introduce MemLife, a multimodal memory system that constructs entity-grounded, first-person text episodes and retrieves them via a time-indexed agentic reader. Without training or query-time video access, MemLife improves over the strongest training-free baseline by 4.6--12.0% across four long-horizon benchmarks. To further improve memory quality, we propose MemOpt, a reinforcement learning framework that optimizes the memory writer to produce faithful, informative, and retrievable memories. MemOpt consistently improves MemLife by 2.7--5.0% across different video and question distributions, with gains that generalize across writer and reader backbones and memory systems.