Llm agents struggle to resume old chats smoothly after finding them
Finding Is Not Returning, Returning Is Not Reconstructing: A Multi-Dimensional Evaluation of Agent Social Re-entry
Artificial IntelligenceSocial and Information Networks
Summary
Conversations with AI agents can last a long time, but these agents often have trouble jumping back into old chats smoothly. This research shows that just remembering past talks isn’t enough; the agents need to find when to return to a chat, decide to reply, and keep the chat coherent. The authors tested different AI setups and found that while some were better at spotting old conversations and deciding to respond, none were consistently good at continuing the conversation naturally. They created a new way to check each of these skills separately.
What this means in practice
- •For customer service teams: Improve chatbot tools to better identify and pick up previous customer conversations without losing context.
- •For virtual assistant developers: Design assistants that can decide when to rejoin paused interactions and keep the conversation coherent over time.
Authors
Sheng Wan, Cong Ma, Sichen Dong, Qian'Ang Mao, Chenhan Zhang, Daning Hu, Qiang Yang
Abstract
As Large Language Model (LLM) agents increasingly participate in persistent social environments, understanding their long-term social interaction capabilities becomes increasingly important. While existing research has advanced long-term memory and conversational context retrieval, less attention has been paid to whether agents can autonomously re-engage with previously inactive conversations. In particular, remembering a past interaction does not necessarily mean that an agent can identify when to revisit it, decide to return, or reconstruct the context needed for a coherent continuation. To investigate this gap, we introduce agent social re-entry as a multi- dimensional evaluation problem, distinguishing three complementary capabilities: Finding, Returning, and Reconstructing. We develop a shared-snapshot evaluation framework that measures these dimensions independently, allowing us to examine how conversational resurfacing and persistent memory relate to different aspects of social re-entry. Evaluations across six LLM configurations show that resurfacing substantially improves target conversation identification, while persistent memory is associated with higher rates of autonomous target-directed replying. However, neither improvement is accompanied by consistent gains in contextual reconstruction. These results highlight the distinction between accessing previous conversations, choosing to re-engage, and maintaining conversational continuity. Our framework provides a way to evaluate these capabilities separately rather than relying on memory retrieval or conversational response quality alone.