Cross-session multimodal memory improves long-horizon task reasoning
C3M: Cross-Session Multimodal Memory Maintenance for Long-Horizon Tasks
Artificial IntelligenceComputation and LanguageComputer Vision and Pattern RecognitionInformation Retrieval
Summary
Long tasks require remembering useful details from past sessions without using too much memory. Existing methods either lose important visual details or mix up similar but different observations. The authors present C3M, a memory system that keeps track of text and images across sessions while managing a fixed memory budget. It carefully updates memory to keep important and compatible information separate, and picks relevant evidence when answering questions. This helps with complex tasks needing information from multiple timed sessions.
What this means in practice
- •For robotics engineers: Maintain relevant visual and textual data across multiple robot operation sessions to improve task completion and decision-making over time.
- •For virtual assistant developers: Manage multimodal memory efficiently over long user interactions to provide consistent and context-aware assistance.
Authors
Xueshu Chen, Yan Wang, Zihao Xue, Jiefu Li, Zhenfang Liu, Jayden Chen, Zhen Bi, Jungang Lou
Abstract
Long-horizon tasks require preserving and later recovering cross-session evidence under a bounded, query-blind memory budget. Existing compression can discard fine-grained visual cues or conflate semantically similar but incompatible observations. We present C3M, a cross-session multimodal memory organization that maintains a bounded active index over persistent source text-image evidence. Relation-aware updates consolidate safe redundancy while preserving complementary and incompatible records. At query time, budgeted routing selects useful index pages and expands their associated source evidence under a fixed reader budget. Together, these mechanisms establish a compact, provenance-preserving multimodal memory organization for cross-session long-horizon tasks, retaining temporal distinctions and source links required for reliable downstream reasoning. Code is available at https://github.com/HuzhouNLP/C3M.