Papers for

customer support platform engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Persistent retrieval cuts costs and speeds up conversational agents

Persistent Retrieval for Conversational Agents: A Study of Retrieval Efficiency

Abstract: Conversational agents can retrieve information directly or delegate retrieval to a subagent. This design choice affects how retrieval history is maintained, what context each model call receives, and what information must be re-retrieved in follow-up questions. We compare three designs: skill-based retrieval in the main agent, a non-persistent retrieval subagent that generates a findings response, and our persistent retrieval subagent, which maintains retrieval history across turns and returns retrieved evidence directly to the main agent. Our approach also allows the main agent to request corrections from the same subagent. We evaluate the three designs on enterprise questions posed independently, within shared conversations, and as dependent follow-ups. Across five multi-turn settings, our approach uses 43.3% fewer recorded agent-model tokens and incurs 45.0% lower estimated API cost than skill-based retrieval, while reducing average response time by 9.1%. Compared with non-persistent retrieval, it reduces response time by 15.6 seconds per question, while using 65.1% more tokens and incurring 29.1% higher estimated API cost. Our approach shifts the retrieval subagent from producing synthesized responses to maintaining retrieval state and supplying evidence that the main agent can directly evaluate and use for answer generation.

Thu 5 NovArtificial IntelligenceComputation and LanguageInformation Retrieval
The gist
Conversational agents answer questions by retrieving information in different ways. The authors compared three designs: one that handles retrieval directly, one that uses a helper without keeping memory, and a new helper that remembers past searches and shares evidence with the main agent. Their new design uses fewer computer resources, lowers costs, and gives faster answers over multiple questions. It works by the helper keeping track of what it found before and providing direct evidence instead of summarizing everything.
Open → 2611.07899v1

Reframe reduces gpu load and speeds up live speech models

ReFrame: Silence-Aware Compute Reclamation for Full-Duplex Speech Models

Abstract: Full-duplex speech models execute dense model inference for every audio frame throughout a live interaction. This uniform framewise schedule creates sustained GPU load across dialogue states whose computation has strong, reusable temporal structure. ReFrame formulates full-duplex inference as a silence-aware computation problem. Mutual Silence Speculation (MSS) operates when both participants are silent: it evaluates multi-frame Temporal continuations, verifies text and hidden-state stability, and applies batched Mimi audio-token correction. Listening Frame Folding (LFF) operates while the user speaks and the agent listens: it processes already-arrived user frames through parallel Temporal execution and frame-batched audio-token generation while accepting only the verified prefix. Frame-Budget Reinvestment (FBR) assigns the released frame intervals to resumable external retrieval and language-model workers, then injects timely references through Moshi's conditioning interface. We instantiate ReFrame on five full-duplex models spanning Moshi-family, phase-structured, and flattened autoregressive generation. Across the complete 1,225-conversation FDBench, ReFrame reduces measured GPU execution time by 18.80-31.01% and retains 96.92-100% non-PAD text agreement on five different models. On HaluEval tests, FBR increases the number of correct generated spoken answers from 86 to 205 under the same auxiliary-service budget. ReFrame turns recurring dialogue structure into lower inference cost and higher end-to-end service capability.

Sun 1 NovMachine Learning
The gist
Full-duplex speech models process every audio frame continuously during a conversation, which uses a lot of computing power. The authors found that by detecting when both people are silent or when one speaks and the other listens, they can skip or batch some calculations without losing accuracy. Their system, ReFrame, smartly reallocates computing time saved this way to improve other tasks like retrieving information. This approach cuts GPU time by up to 31% while keeping almost all text outputs correct and also helps increase correct spoken answers.
Open → 2611.01835v1

Unified model improves text embedding and generation together

ULME: Predictive Coupling for Mutually Reinforcing Embedding and Language Modeling

Abstract: Text embedding and text generation are two of the most useful capabilities of modern large language models, yet they are typically provided by separate specialized models (an embedder and a generator) trained with incompatible objectives and deployed side by side in retrieval-augmented generation and agent systems. Recent unified embedding-generation models such as GritLM share parameters between the two capabilities, but they either switch attention regimes between modes or optimize embedding and language modeling as parallel objectives that never supervise each other. We argue that these designs are only partially unified: the two capabilities coexist on the same weights, but do not shape each other. We propose ULME, a post-training framework that unifies embedding and language modeling on a causal decoder-only LLM through predictive coupling. ULME inserts a chain of learnable slots between the query and the answer, reads a semantic embedding and a reasoning embedding from two designated slots, and trains three objectives on a shared causal backbone: a contrastive loss anchoring the semantic slot to document representations, a language-modeling loss supervising answer generation, and a stop-gradient predictive loss that aligns the reasoning slot with a pooled summary of the answer, so that generation actively shapes what the embedding encodes. Across two model families (Qwen2.5, Qwen3) and eight scales spanning 0.5B-8B parameters, a single ULME checkpoint improves MTEB by 3.3-4.1 points over the strongest open-source baseline, tightens BRIGHT and AdvBench-IR simultaneously, and remains on par with a generation-only fine-tune on standard accuracy benchmarks (MMLU, GSM8K, MMLU-Pro, HumanEval), demonstrating that genuine unification yields mutually reinforcing rather than merely coexisting capabilities.

Thu 22 OctComputation and LanguageArtificial Intelligence
The gist
Text embedding and text generation are usually handled by separate models, which do not influence each other. The authors of this paper propose ULME, a way to train a single model that combines these two tasks so they help each other improve. ULME inserts special slots in the model that let it learn embeddings based on the text it generates, creating a stronger connection between understanding and generating text. Tests on different model sizes show ULME improves embedding quality without sacrificing generation accuracy.
Open → 2610.30705v1

Large language models hold hidden clues to pinpointing failures

A Mechanistic Study of Automated Failure Attribution in LLM-based Agentic Systems

Abstract: Agentic systems have demonstrated strong capabilities on complex tasks by enabling multiple interacting large language model (LLM)-based agents to collaborate and execute subtasks in parallel. However, this collaborative structure introduces new challenges for reliably evaluating and diagnosing system failures, particularly failure attribution: tracing a system's execution to identify the agent step responsible for an incorrect final outcome. Because manually performing this task is time- and labor-intensive, recent work has turned to LLMs as judges to automate the process, referred to as automated failure attribution (AFA). These approaches assume that a sufficiently capable model can infer the failing step directly from an interaction transcript. In practice, however, direct prompting often performs poorly, which motivates follow-up work that explores various ways to improve AFA performance, including alternative prompting strategies and inference-time interventions. In this paper, we examine a more fundamental question: whether this weak behavioral performance reflects a genuine capability limitation. Using linear probes applied to internal activations, we uncover a significant \emph{knows-but-won't-say gap:} models encode sufficient information to attribute a failure correctly but fail to explicitly output the responsible step. Building on this finding, we propose \emph{grafting}, a simple manual intervention that exploits knowledge already encoded in the model to improve AFA performance without requiring ground-truth labels at inference time. Extensive evaluations across multiple LLMs and benchmark settings demonstrate that grafting consistently outperforms direct-prompting approaches, underscoring the potential of leveraging latent model knowledge for more reliable failure attribution. Our code is available at https://github.com/wendyzheng22/mi4afa.

Tue 20 OctMachine Learning
The gist
Figuring out exactly where a group of AI agents goes wrong on a task is tricky and usually needs a lot of human work. The authors found that while AI models often don’t explicitly say which step caused a failure, they actually have the information hidden inside. They introduced a simple way to uncover this hidden knowledge to better identify failure points without needing extra labeled examples. This helps improve how we can automatically find mistakes in AI systems working together.
Open → 2610.28811v1

Memory agents improve answers by learning from process rewards

All-in-One Memory: Process-Rewarded Reinforcement Learning for Memory Agents

Abstract: Large language models increasingly rely on external memory to answer queries grounded in long-horizon histories such as multi-session dialogues and lengthy documents. However, existing memory methods often optimize retrieval and memory construction separately, without a shared reward space for evaluating search, writing, and stopping decisions. To address this, we propose All-in-One Memory (AiO-Mem), which organizes these operations as transitions over a shared evidence state and directly evaluates the semantic quality of each action in a unified reward space. AiO-Mem compiles heterogeneous histories into source-traceable multi-view events, providing a shared substrate for retrieval and memory writing. A unified memory policy actor then rolls out complete interaction trajectories over this shared evidence state. A separate trajectory-conditioned process reward model is trained through evidence-grounded supervised fine-tuning and trajectory-level DPO to assign process scores to the collected trajectories. These scores are aggregated into an action-balanced trajectory reward and used by Process-Aligned DAPO (PA-DAPO) to optimize the memory policy. Across long-context memory benchmarks, AiO-Mem consistently improves answer accuracy over both static memory systems and RL-trained memory agents.

Thu 15 OctArtificial Intelligence
The gist
Long conversations and documents are hard for AI to remember and use well because their memory processes are often handled separately. The authors propose a new method that treats remembering, searching, writing, and stopping as one connected process with shared rewards. This method helps AI keep better track of important information and decide what to remember or retrieve. It was tested on tasks requiring understanding long contexts, showing it improves answer accuracy compared to older methods.
Open → 2610.21147v1

Combining natural language and modular prompts for adaptable ai alignment

Ready2Blend: From Natural-Language Instructions to Composable Alignment Prompts

Abstract: Continual alignment requires LLMs to adapt to new requirements without forgetting previously acquired behaviors. Natural-language instructions are flexible and composable but offer only indirect control, whereas post-training provides stronger adaptation at the cost of repeated parameter updates. We introduce Ready2Blend, which combines the flexibility of natural language with learned alignment. AlignFormer maps each requirement to a fixed-length alignment prompt stored in a modular prompt bank, while the backbone and prior prompts remain frozen. Composability regularization transfers the semantic geometry of textual requirements into prompt space, enabling inference-time blending and reweighting. Across two practical continual alignment settings, Ready2Blend is the only frozen-backbone method that matches post-training-based alignment methods, reaching $93.1$-$98.5\%$ of a joint-training reference with competitive retention, while requiring only a few prompt tokens and up to $4.3\times$ less training time. Its modular design further enables weighted personalization and order-free composition without retraining. Code will be released upon acceptance.

Wed 30 SeptComputation and LanguageArtificial Intelligence
The gist
Getting large language models (LLMs) to follow new instructions without forgetting old ones is hard. The authors created Ready2Blend, which uses special prompts linked to text instructions, letting models adapt by mixing these prompts without changing their core. This method works almost as well as retraining the model fully but is faster and more flexible, supporting personalized and combined instructions without extra training. It keeps the main model unchanged, speeding up updates while keeping past skills.
Open → 2609.39365v1

Llms often use outdated information despite having updates available

When Context Changes: Understanding Update Failures in LLMs

Abstract: As preferences, goals, and facts change, LLM agents must use the current state while earlier versions remain in context. Yet they can answer with an old value of the same variable, a failure that we call stale binding. To study when models use outdated information and why, we introduce Controlled In-Context Memory (CICM), a benchmark for tracking and using updated information in conversations and agent logs. We observe that even frontier reasoning models can fail to recover the current state. We find that in open-source models probes can still recover the updated value when the model answers with an old one, pointing to a failure to select information that remains available. Component tests in Qwen and Pythia identify a mechanism for this selection failure: attention drift, where attention favors old values over the current one when producing an answer. We study a one-layer transformer to mathematically understand how this phenomenon happens: when attention scores are similar, several old values can together receive more attention than the current value. Guided by this explanation, we redirect attention toward the current value without further training. When the current value is requested directly, adjusting this intervention for each input corrects most old-value errors across various model families while preserving nearly all initially correct answers. Reliable context management therefore requires more than remembering updated information: models must use it to guide their answers.

Wed 30 SeptArtificial Intelligence
The gist
Large language models (LLMs) sometimes answer questions using old information even when newer updates are in their context. The authors call this problem stale binding. They created a tool named Controlled In-Context Memory (CICM) to track when models fail to use the latest facts. Their research found that this happens because the model’s attention focuses more on older information, not because the updates are missing. They developed a way to redirect attention to the latest facts, which greatly reduces these errors without retraining the model.
Open → 2609.38866v1

Progressive disclosure improves agent skills quality but adds delay

Report: Progressive Disclosure of Agent Skills

Abstract: Users of Workday's deployed LLM-based agents often request features which can be addressed by defining named procedures, also known as skills, in the LLM context, effectively augmenting agents' capabilities. However, as an agent's skills library grows in size, so does the agent's operational cost. Progressive disclosure (lazy-loading) of skills as needed may reduce operational costs, but its impact on overall latency and skill-retrieval quality remains unclear. In this report, we investigate the impact empirically and find that progressive disclosure improves skill-retrieval quality but marginally degrades overall latency.

Mon 28 SeptArtificial Intelligence
The gist
Large language model agents can do more things when given extra skills, but having too many skills makes them expensive to run. The authors studied turning on skills only when needed, called progressive disclosure, to save costs. They found this approach makes it easier for the agent to pick the right skill but slightly slows down how fast it answers. So, adding skills gradually helps quality but at a small speed cost.
Open → 2609.35692v1

Stashbird cuts memory tokens for ai agents with indexed updates

Stashbird: Efficient Speaker-Indexed Memory for Conversational Agents

Abstract: AI agents require memory that preserves information across user-agent exchanges, user-to-user conversations, and group conversations with or without agent participation, while supporting updates as evidence changes or is removed. We present Stashbird, an agent memory system that links source episodes to derived memory state through explicit provenance. Stashbird organizes memory into episodic records, semantic relations, community summaries, and persisted graph state, with lifecycle operations for incremental updates and episode-level deletion. We evaluate question-answering accuracy and model-facing workload across four long-term memory benchmarks. On LoCoMo, Stashbird uses 76.4x fewer ingestion prompt tokens than Graphiti. Compared with reproduced Hindsight on the same benchmark, it uses 8.1x fewer retrieval prompt tokens, with accuracy 1.6 percentage points lower. It achieves higher accuracy than Hindsight on LongMemEval-S and GroupMemBench and comparable accuracy on EverMemBench.

Mon 28 SeptArtificial IntelligenceMachine Learning
The gist
AI assistants need to remember past conversations accurately, including who said what and when information changes. The authors created Stashbird, a system that organizes memory with clear links to original conversations and allows easy updates or deletions. This system uses much fewer resources to process and recall information than some existing methods, while maintaining similar or better accuracy. Stashbird is tested across multiple benchmarks involving long-term conversational memory.
Open → 2609.34242v1