Papers for

virtual assistant teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Memory system improves character dialogue by tracking mental states

We Are What We Remember: Mental-State-Driven Memory for Long-Term Character Dialogue

Abstract: Shared history in human dialogue is not merely a repository of facts, but the foundation of evolving relationships and mutual understanding. However, mainstream conversational memory systems rely on passive semantic similarity over external fact stores, frequently retrieving topically similar records while missing relationally pivotal experiences. Inspired by cognitive psychology's Self-Memory System (SMS), we propose MindMem, a mental-state-driven memory framework that asynchronously consolidates shared experiences into source-linked mental states of the character, the user, and their relationship. At each turn, a specialized mental-state reasoner (Mind) contextualizes these persistent states to anticipate latent user needs, form proactive intentions, and recover supporting episodic evidence that surface-level matching overlooks. To evaluate history-sensitive adaptation beyond factual recall, we introduce MemReal-Bench, a longitudinal benchmark grounded in month-long real-user interactions across dyadic and group scenes. By pairing Direct Question Answering (DQA) with Situated Memory Realization (SMR), it tests whether agents bridge the gap between knowing past facts and naturally using shared history, world rules, and relational trajectories. On MemReal-Bench, MindMem reaches 59.03% SMR and 82.58% DQA, exceeding the strongest memory baseline by 24.19 and 9.19 percentage points, respectively, and outperforming oracle target-memory injection on SMR. Code and dataset are available at https://github.com/jiazhicheng7-ctrl/MindMem.

Thu 12 NovComputation and Language
The gist
In human conversations, remembering shared experiences helps build relationships, not just facts. The authors created MindMem, a system that mimics how people mentally store memories about themselves, others, and their relationships, to better recall important past interactions. This system uses these mental states to guess what a user might need and find relevant memories beyond simple fact matching. They tested it on a month-long benchmark with real users and found it improved dialogue understanding and response accuracy.
Open → 2611.16223v1

Re ranker improves candidate scoring with reasoning based queries

Re$^3$Rank: Listwise Reranking with Reasoning-Conditioned Retrieval Heads

Abstract: Attention-based rerankers score each candidate by the attention mass that retrieval heads route to it from the question, but the question, serving as a fixed query, carries limited information for locating relevant evidence. We propose Re$^3$Rank, a listwise reranker built on reasoning-conditioned retrieval heads (RRHeads). A generator first writes a reasoning trace over the question and the candidates, and a scorer then uses the trace tokens as the attention query of a few selected RRHeads, reading relevance as continuous scores. We present a straight-through estimator to propagate the ranking loss through the discrete trace, so that the generator and the scorer can be trained jointly for ranking. Averaged over 9 benchmarks, Re$^3$Rank outperforms 11 baseline rerankers: our 4B variant exceeds the strongest baseline by 3.11 NDCG@10 points on average, and even our 0.6B variant surpasses rerankers with up to 32B parameters.

Mon 2 NovComputation and Language
The gist
Ranking answer choices usually relies on just the question, which limits how well relevant information can be found. The authors propose a new system called Re$^3$Rank that first creates a step-by-step reasoning trace combining the question and answer options. Then it uses this reasoning to better focus on important parts when scoring each candidate. This joint approach helps the system pick better answers and outperforms many existing methods across several benchmarks.
Open → 2611.02550v1

Vision language action models struggle to follow how instructions

Do VLAs Really Follow Instructions? Diagnosing How-Level Instruction Following in Vision-Language-Action Models

Abstract: Vision-language-action (VLA) models are expected to follow instructions like their language backbones, yet prior work has focused on What-level instructions that specify the task rather than How-level instructions that specify how to carry it out. Through our diagnoses, we find that How-level instructions do not help current VLAs and lower task success. We hypothesize that pairing each training episode with a single instruction limits How-level instruction following, and show that training with segment-level enriched instructions makes actions respond to such instructions without consistently raising task success. To measure How-level instruction following beyond standard benchmarks, we introduce SIMPLER-IN, where success requires conditions given only in the instruction. It shows that the gained responsiveness does not extend to demanding or out-of-distribution How-level instructions. How-level instruction following thus remains an open challenge.

Thu 15 OctComputer Vision and Pattern RecognitionRobotics
The gist
Many AI systems that combine vision and language are expected to follow detailed instructions on how to perform tasks. This paper finds that current models do not improve when given step-by-step (how-level) instructions and often perform worse. The authors suggest that training methods limit the models' ability to respond to such instructions, even when attempts are made to improve it. They introduce a new test that better evaluates following how-level directions, showing that current models still fail on harder or unexpected instructions. So, being able to follow detailed instructions remains a challenge for these systems.
Open → 2610.21251v1

Multimodal agent memory evolves for long tasks and changing environments

Seeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memory

Abstract: Multimodal models and agents have emerged to enable AI systems to learn from and interact with the real world more like humans by integrating information across multiple modalities. However, enabling multimodal agents to continually acquire useful information from complex, heterogeneous multimodal data remains a significant challenge as they move toward long-horizon tasks and lifelong settings. Addressing this challenge requires effective memory mechanisms that can maintain continuity across sessions, adapt to changing users and environments, and refine memory-management policies through accumulated experience. We introduce a two-axis analytical framework for multimodal agent memory structured around two key questions: how multimodal information is formed and represented (Axis I, Formation and Representation), and how its management adapts with accumulated experience (Axis II, Memory Management). Axis I, Formation and Representation, characterizes what information is retained and how it is encoded (e.g., modality-specific embeddings, textual and structured records, latent and parametric representations) to support subsequent reasoning and action. Axis II, Memory Management, distinguishes three tiers: T1 Static Memory reuses fixed memory; T2 Dynamic Memory updates memory through a stable policy; and T3 Self-Evolving Memory updates memory through an evolving policy. Based on this framework, we synthesize recurring design patterns and derive principles for balancing source fidelity with abstraction, maintaining temporal validity and provenance, grounding memory use in recoverable evidence, and evolving memory policies over time. Furthermore, we discuss the challenges and potential future directions for the memory mechanism of multimodal agents.

Wed 14 OctArtificial IntelligenceComputation and LanguageRobotics
The gist
AI systems that work with multiple types of information, like images and text, need good memory to remember important things over time and adapt to new situations. The paper organizes different ways to store and manage this memory into two categories: how the information is saved and how the system updates its memory strategies based on experience. It explains various methods, such as keeping fixed memories or evolving memory policies as the system learns. The authors also discuss challenges and ideas for improving how these AI agents use memory for tasks lasting a long time.
Open → 2610.19366v1

Object oriented context improves long horizon agent efficiency

Pack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agents

Abstract: Long-horizon agents accumulate progress in persistent task state, yet repeatedly process interaction histories describing how that progress was achieved. This raises a natural question: can long-horizon context instead be organized around reusable progress rather than the history that produced it? In this paper, we introduce CarryOn, an object-oriented context runtime that organizes continuation around reusable task objects. Rather than treating compressed histories or reconstructed accounts of progress as the primary basis for continuation, CarryOn builds a context view from continuation carriers, complementary execution context, and a live tail of recent interactions. A core challenge is keeping this reusable basis valid after its surrounding history is retired: how can CarryOn update the basis without reconstructing the full representation at every fold? CarryOn maintains carrier records across folds, revising affected records against bound evidence and applicable environment checks while carrying forward unaffected records and jointly updating execution context. Experiments on OSWorld 2.0 and BrowseComp-Plus show improved gains of 5.8% and 7.0% while reducing total token usage by 51.9% and 22.9%, respectively. Project page: https://ethannju.github.io/carryon

Tue 13 OctArtificial IntelligenceComputation and Language
The gist
Long-running AI agents need to remember what they have done to continue tasks well, but storing all past actions can be slow and costly. The authors introduce CarryOn, a system that organizes the agent’s memory around reusable task objects instead of long histories. CarryOn updates these objects efficiently to keep them valid without reprocessing everything repeatedly. Tests show CarryOn helps agents perform better while using fewer computing resources.
Open → 2610.18784v1

Large language agents fail to load relevant skills when overloaded with many options

Lost in Implicit Routing: How LLM Agents Fail to Load Relevant Skills at Scale

Abstract: Reusable skills let LLM agents load task-specific instructions on demand, but they make execution depend on implicit routing over skill metadata. We test whether this design scales by holding SkillsBench tasks and their golden skills fixed while adding irrelevant skills. We find skill routing collapse: agents increasingly load fewer or no skills, even though the relevant skills remain available. With 1,000 irrelevant skills, loaded-skill recall falls from 73.6% to 27.4% and the zero-load rate rises from 9.3% to 51.6%, whereas explicit selection from the same library retains 73.9% recall. Loading other skills, including plausible alternatives to golden skills, is too rare to explain this decline. Masking only the descriptions of irrelevant skills largely restores skill loading, yet with the metadata intact, explicitly asking agents to plan skill use still yields lower recall than preselection. These findings point to semantic interference from descriptions and instruction interference when skill choice competes with task execution. We also evaluate explicit preselection as a practical mitigation, including its coverage and latency trade-offs. Our analysis informs skill protocols and evaluation for agents with growing skill libraries.

Tue 13 OctArtificial Intelligence
The gist
When AI agents try to pick the right skills from a big list to solve tasks, they often fail if there are too many unrelated skills mixed in. The researchers found that agents load fewer or no relevant skills as the skill library grows, even though the right skills are still there. The problem seems to come from confusing skill descriptions and instructions competing with task execution. However, explicitly selecting skills beforehand helps fix this issue.
Open → 2610.18522v1

Memory echo correction boosts single-agent long-term accuracy significantly

Memory Echo in Single-Agent Long-Term Memory: Lifecycle Evaluation and Provenance-Guided Mitigation

Abstract: Long-term memory supports agents across sessions. Repeated retrieval and write-back can let one historical claim occupy multiple retrieval positions and influence current answers without independent evidence. Existing work examines retrieval, updating, and question answering, leaving same-source record accumulation and post-correction persistence underexplored. Motivated by research in social cognition and memory psychology on repeated exposure, retrieval, and revision, we define \emph{Memory Echo} as the excess retrieval exposure gained by historical information from the same source without additional independent evidence, and introduce MEchoBench to evaluate its emergence, correction resistance, persistence, and effect on answers. To address this phenomenon, we introduce the Family--Provenance--Authority framework (FPA). Specifically, FPA selects one representative per source family, conditions the eligibility of old records for the answer context on verified corrections, and constrains final answers with authoritative evidence. In paired replay on 190 fixed questions across four memory frameworks, FPA raises accuracy from 0.825 to 0.920 with DeepSeek V4 Flash and from 0.782 to 0.916 with Kimi K2.6; mean Decision Contamination Effect falls from 0.106 to 0.030 and from 0.109 to 0.038, respectively. These results show that agent memory can preserve traceable history without allowing repeated historical information to govern current decisions. The code is available at https://github.com/OrcasSam/memory-echo-benchmark.

Tue 13 OctArtificial Intelligence
The gist
When computer agents remember information over a long time, repeated recalling of the same old facts can unfairly make those facts seem more important, even if there's no new proof. The authors call this problem Memory Echo, which can cause mistakes in answers. They created a test called MEchoBench to see how often this happens and how hard it is to fix. To solve it, they designed a method called FPA that picks one main fact from each source, checks corrections carefully, and relies on trusted evidence. This improved the accuracy of agents’ answers quite a bit without losing useful memory.
Open → 2610.17763v1

Reinforcement learning improves social understanding without labeled data

Reinforcement Learning with Comparative Evidence for Social Intelligence

Abstract: Developing socially intelligent AI remains heavily dependent on human-annotated data, limiting the scale and breadth of social understanding models can acquire. Methods that derive training signals from unlabeled data offer a path beyond this dependence, but social predictions lack the verification oracles available in mathematics and coding. Moreover, core social targets such as affect, intent, preference, and pragmatic meaning are often ambiguous. The same behavior can support multiple plausible interpretations, making it difficult to verify which is best supported. To address this challenge, we introduce Reinforcement Learning with Comparative Evidence (RLCE), a reinforcement learning method that learns social understanding from unlabeled training data without constructing rewards from ground-truth annotations. Given distinct answers in a rollout group, RLCE constructs evidence tests that identify observable evidence favoring an answer over another, validates these tests against the input sample, and aggregates test outcomes to determine the best-supported interpretation. Tests are regenerated as the policy produces new answers, enabling them to evolve with the policy. Across four benchmarks spanning affect, pragmatics, communicative intent, and preference, RLCE attains the strongest performance among seven methods that use no ground-truth training labels for rewards, including consensus, policy LLM-judge verification, multimodal co-evolution, and rubric-based rewards. Gains over the strongest baseline reach up to +18.93 points. Analyses further show that RLCE exhibits a larger share of reward variation between correct and incorrect predictions than compared rubric methods, can overturn erroneous policy-derived preferences, and benefits from pairwise test construction, compositional test aggregation, and on-policy test evolution.

Fri 2 OctArtificial Intelligence
The gist
Teaching AI to understand social signals often needs lots of labeled examples from people, which is hard to get. The authors introduce a method called RLCE that helps AI learn social cues like emotions and intentions from unlabeled data by comparing different possible answers. RLCE uses tests that evolve as the AI improves to find which interpretation fits best, even when behaviors can be ambiguous. This approach worked better than other similar methods on tasks involving feelings, intent, and preferences.
Open → 2610.04072v1

Llms often use outdated information despite having updates available

When Context Changes: Understanding Update Failures in LLMs

Abstract: As preferences, goals, and facts change, LLM agents must use the current state while earlier versions remain in context. Yet they can answer with an old value of the same variable, a failure that we call stale binding. To study when models use outdated information and why, we introduce Controlled In-Context Memory (CICM), a benchmark for tracking and using updated information in conversations and agent logs. We observe that even frontier reasoning models can fail to recover the current state. We find that in open-source models probes can still recover the updated value when the model answers with an old one, pointing to a failure to select information that remains available. Component tests in Qwen and Pythia identify a mechanism for this selection failure: attention drift, where attention favors old values over the current one when producing an answer. We study a one-layer transformer to mathematically understand how this phenomenon happens: when attention scores are similar, several old values can together receive more attention than the current value. Guided by this explanation, we redirect attention toward the current value without further training. When the current value is requested directly, adjusting this intervention for each input corrects most old-value errors across various model families while preserving nearly all initially correct answers. Reliable context management therefore requires more than remembering updated information: models must use it to guide their answers.

Wed 30 SeptArtificial Intelligence
The gist
Large language models (LLMs) sometimes answer questions using old information even when newer updates are in their context. The authors call this problem stale binding. They created a tool named Controlled In-Context Memory (CICM) to track when models fail to use the latest facts. Their research found that this happens because the model’s attention focuses more on older information, not because the updates are missing. They developed a way to redirect attention to the latest facts, which greatly reduces these errors without retraining the model.
Open → 2609.38866v1

Language agents that communicate effectively with fewer words

Towards Communication-Efficient Social Intelligence in Language Agents

Abstract: Socially intelligent language agents must negotiate, coordinate, and resolve conflicting preferences while respecting the time and attention of both participants. Balancing these demands is challenging because agents must convey enough to address a partner's constraints and advance their goals without adding words that do not help the interaction. In this paper, we propose Teacher-Assisted Communication Training (TACT) to improve social goal attainment while reducing communication cost, making interactions with agents more productive and less demanding. We first characterize communication efficiency in terms of action strategy and expression, whose effects extend beyond the current utterance to the partner's response and subsequent exchanges. We design TACT to revise student-generated actions, test the revisions through partner responses, and distill useful feedback into the student. An expression specialist removes unnecessary detail while preserving the intended action, while a strategy specialist proposes alternatives that may better address the partner's constraints. To determine which revision helps, TACT samples a partner response for each candidate and selects a teacher reference by balancing local goal support against action-token cost. That reference guides on-policy distillation on the student's own generation prefixes, allowing the student to act independently at deployment. We evaluate TACT on SOTOPIA and AgentSense. On SOTOPIA, it achieves the highest Goal among the evaluated methods on All and Hard while using substantially fewer target tokens than SFT+SDPO. On AgentSense, it improves goal success over the initial student while reducing target tokens and interaction messages.

Mon 28 SeptComputation and Language
The gist
Socially smart language agents need to cooperate by sharing just the right amount of information, so conversations stay useful without being long or confusing. The authors created a method called Teacher-Assisted Communication Training (TACT) that teaches these agents to pick their words and actions carefully, helping them meet goals while using fewer words. TACT works by checking different ways of saying something, guessing how the partner might respond, and then learning which version works best with the smallest cost in words. The authors tested TACT in two settings and found it improved success rates while cutting down on unnecessary communication.
Open → 2609.35749v1

Models often fail to forget wrong user claims after reset

Reset Is Not Recovery: Evaluating Recoverability from False Conversational Context via Sycophancy Hysteresis

Abstract: Grounded language models are usually evaluated by adding relevant context, but multiturn dialogue also contains unsupported user claims that may contaminate later factual answers. We study post-pressure recoverability: whether a model returns to clean-context behavior after a user repeatedly advocates a wrong answer and then withdraws that pressure. We introduce a recovery-after-pressure protocol for multiple-choice factual dialogue and measure sycophancy hysteresis, the residual probability assigned to the user-advocated wrong answer relative to a clean-context counterfactual. Across seven instruction-tuned open-weight models and two factual benchmarks, ordinary reset often reduces but does not erase pressure-induced bias. History preserving repairs such as user retraction, system reset, and self-verification recover only 2-3/14 model-dataset pairs under the strict clean-restoration diagnostic, whereas operations that change the effective context are substantially more reliable; the two conditions that remove the pressure-bearing history entirely, fresh-context deletion and context truncation, recover 14/14. In an oracle trusted-evidence condition across fourteen model-dataset pairs, preserving the pressure-bearing history while adding benchmark-derived trusted evidence increases accuracy from 0.368 to 0.929, while wrong-answer following falls from 41.2% to 4.3%. Controls show that the effect is not explained by dialogue length, repeated confidence, plausible distractors, mere false-answer mention, or option-label inertia. These results suggest that faithful grounded dialogue requires evaluating which prior context should be treated as evidence and which should be removed or quarantined before answering.

Sun 27 SeptComputation and Language
The gist
Sometimes AI chat models are told wrong facts by users during a conversation, and later they try to correct themselves. This paper studies whether these models can truly recover and stop believing the wrong information, once the user admits the mistake. The authors found that simply resetting the conversation often doesn't fully remove the wrong influence. Only completely removing or truncating the chat history reliably restores the model to giving correct answers. Adding trusted evidence to the conversation greatly improves the model’s accuracy even if previous wrong claims remain.
Open → 2609.33672v1

Persona mixture models improve simulation of real human dialogue

Pretrained Persona Mixture Models and Tandem Models for Human Simulation

Abstract: We argue here that the current dominant practice in LLM human simulation: prompting instruction-tuned assistant language models to role-play personas, is inaccurate and produces stereotyped predictions (lacking natural diversity). It has previously been shown that LLMs can be bound to personas using naturalistic, freetext dialog avoiding stereotyping. Here we show that binding can also be achieved using short, individual samples of dialog from specific people. Demographics can be added later without negative effects by simply querying the model. We use the term Persona Mixture Models (PMMs) for well-calibrated human models, currently realized as pretrained base models. We show that PMMs produce more accurate predictions than instruction-tuned models and retain more of the lexical, semantic, and pragmatic diversity found in human dialog. We measure realism and diversity of LLMs simulating human interlocutors across a diverse set of corpora spanning open-domain text, human-AI chat, and task-oriented dialogue between human speakers. However, base pretrained models can produce out-of-domain dialog and may lose some of the human's internal state over long contexts. We propose and explore tandem models which combine a pre-trained model with an instruction-tuned supervisor. Tandem models achieve the best overall accuracy and diversity in our experiments.

Fri 18 SeptComputation and Language
The gist
Simulating how people talk using AI often leads to unrealistic and repetitive answers when the AI just pretends to be a certain character. The authors show that training AI on actual short conversations from real people helps the AI respond in a way that sounds more natural and diverse. They call these trained models Persona Mixture Models and find they work better than models just tuned with instructions. Combining these models with other instruction-following models makes the best predictions over different kinds of conversations.
Open → 2609.22607v1

Automated views improve memory retrieval in long conversations

AutoViewMem: Self-Configuring Orthogonal Views for Conversational Long-Term Memory

Abstract: Long-term memory is essential for large language model (LLM) agents to maintain consistency and personalization over extended interactions. Existing memory systems typically rely on fixed granularities or static schemas, but these designs struggle when heterogeneous information, such as preferences, events, constraints, and temporal updates, is embedded in a single mixed representation. The resulting semantic interference makes top-K retrieval sensitive to noise and often leaves relevant evidence poorly ranked. We present AutoViewMem, a data-driven framework that organizes long-term conversational memory into self-configuring, low-overlap semantic views before indexing. AutoViewMem discovers candidate views from interaction traces, selects a compact complementary view set, and uses these views to guide write-time structured extraction of provenance-grounded memories. This representation-first design moves semantic disentanglement from retrieval time to write time, allowing standard top-K similarity search to retrieve focused evidence without explicit routing or iterative retrieval. We further apply offline consolidation to improve memory compactness and consistency. Experiments on the LoCoMo and PersonaMem benchmarks, under both Qwen3-8B and Qwen3-14B backbones, show that AutoViewMem improves long-horizon question answering and personalization over strong memory baselines while preserving a simple inference pipeline.

Fri 18 SeptArtificial Intelligence
The gist
Remembering details accurately is hard for AI chatbots during long talks. The authors show a way to organize the chatbot’s memories into separate groups based on topics, so it doesn’t mix up different types of information. This helps the AI find the right facts more easily when answering questions or personalizing chats. The system they created works better than older methods and keeps things simple when the AI responds.
Open → 2609.21940v1

GLARE improves forecasting of conversation flow in real meetings

GLARE: Generative Learning via Adversarial Reward Estimation For Social Dynamics Forecasting

Abstract: Meeting continuation requires tracking the agenda, speaker roles, participant intentions, and disagreement across long multi-party discussions. We introduce the Meeting Dynamic Forecasting Benchmark (MDFB), constructed from 2,207 real-world meetings and 24,794 future-facing queries. Given a transcript prefix and an active question, a model generates a plausible multi-turn continuation in one call. We evaluate utility---progress toward the question---and human-likeness---plausible conversational flow and role consistency---without requiring exact reproduction of the observed future. We further present GLARE, an adaptation of adversarial imitation learning to conditional language generation. A discriminator ranks the observed continuation above samples from the current actor, and its score supplies a KL-regularized policy reward; retraining on current-policy negatives allows the reward landscape to evolve with the actor. GLARE attains average human-evaluated win rates of 0.66 on utility and 0.70 on human-likeness, outperforming SFT and SPIN while remaining below the observed human continuation. We also demonstrate MDFB as a social reasoning arena for comparing general-purpose models, including closed-source systems, through reference-assisted judgments. Together, these studies illustrate the benchmark's use for both task-specific learning and output-based evaluation of meeting behavior.

Thu 10 SeptArtificial Intelligence
The gist
Following the flow of real meetings is hard because people talk, have roles, and change their minds over time. To help with this, the authors created a big set of meeting recordings and questions about what happens next, called MDFB. They also built a method called GLARE that teaches computers to guess what might happen later by comparing guesses to real meeting parts and learning from that. GLARE does better than earlier methods at making useful and natural-sounding meeting continuations.
Open → 2609.12165v1