Memory system improves character dialogue by tracking mental states

We Are What We Remember: Mental-State-Driven Memory for Long-Term Character Dialogue

Computation and Language

Summary

In human conversations, remembering shared experiences helps build relationships, not just facts. The authors created MindMem, a system that mimics how people mentally store memories about themselves, others, and their relationships, to better recall important past interactions. This system uses these mental states to guess what a user might need and find relevant memories beyond simple fact matching. They tested it on a month-long benchmark with real users and found it improved dialogue understanding and response accuracy.

What this means in practice

  • For chatbot developers: Design dialogue agents that remember relational history, enabling more natural and context-aware conversations with users over long periods.
  • For virtual assistant teams: Build assistants that proactively anticipate user needs by reasoning over evolving shared memories and mental states across interactions.

Authors

Zhicheng Jia, Shuchen Wu, Zhuoshi Tong, Haisen Liao, Sicheng Zhou

Abstract

Shared history in human dialogue is not merely a repository of facts, but the foundation of evolving relationships and mutual understanding. However, mainstream conversational memory systems rely on passive semantic similarity over external fact stores, frequently retrieving topically similar records while missing relationally pivotal experiences. Inspired by cognitive psychology's Self-Memory System (SMS), we propose MindMem, a mental-state-driven memory framework that asynchronously consolidates shared experiences into source-linked mental states of the character, the user, and their relationship. At each turn, a specialized mental-state reasoner (Mind) contextualizes these persistent states to anticipate latent user needs, form proactive intentions, and recover supporting episodic evidence that surface-level matching overlooks. To evaluate history-sensitive adaptation beyond factual recall, we introduce MemReal-Bench, a longitudinal benchmark grounded in month-long real-user interactions across dyadic and group scenes. By pairing Direct Question Answering (DQA) with Situated Memory Realization (SMR), it tests whether agents bridge the gap between knowing past facts and naturally using shared history, world rules, and relational trajectories. On MemReal-Bench, MindMem reaches 59.03% SMR and 82.58% DQA, exceeding the strongest memory baseline by 24.19 and 9.19 percentage points, respectively, and outperforming oracle target-memory injection on SMR. Code and dataset are available at https://github.com/jiazhicheng7-ctrl/MindMem.