Papers for

customer support platforms

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Statewise improves language models tracking and obeying dialogue rules

StateWise: Teaching Language Models to Track and Obey Evolving Constraints in Multi-Turn Dialogue

Abstract: Users of conversational assistants rarely state all requirements up front: they add constraints, amend them, withdraw them, digress, and return. Instruction-tuned LLMs fail here in two ways: active constraints stop being honored as turns accumulate (persistence failure), and withdrawn constraints keep being enacted (relapse). The only existing repair we know of is an external ledger that re-injects a compiled specification every turn. We propose StateWise, a self-improving approach that teaches a single LLM to maintain its own revisable constraint state, a typed state with tombstones for revoked constraints, so persistence and relapse are measured by executable checkers and ground-truth states are compiled for free. StateWise fine-tunes the LLM on two complementary skills, summarizing a dialogue into its constraint state and responding conditioned on that state, then self-trains on its own trajectories, keeping only turns that pass an exact round-trip check against the compiled state. This teaches 7-8B models to track the state almost perfectly but, surprisingly, not to obey it: on Llama-3.1-8B every supervised variant roughly doubles relapse. We therefore add a preference stage on self-generated relapse contrasts, where the rejected response enacts a revoked rule and the chosen one passes the round-trip check. On two backbones with held-out constraint families and unseen phrasings, StateWise cuts relapse by 8-22 points relative to supervised self-training, lifts conversation success by 8-19 points over the backbone, and beats an oracle compiled-ledger baseline on relapse with no inference-time scaffolding. We also characterize the recipe's cost on general instruction-following benchmarks and study weight interpolation as a mitigation.

Sat 24 OctComputation and LanguageArtificial IntelligenceMachine Learning
The gist
Conversational assistants often struggle to remember changing user requests during a chat, sometimes forgetting rules or wrongly applying canceled ones. The authors propose StateWise, a method that teaches a language model to keep an updated list of rules it must follow, including tracking which ones are removed. They fine-tune the model to summarize and respond based on these changing rules, and further improve obedience by teaching it to prefer responses that follow updated instructions. This approach significantly reduces the mistakes where the assistant ignores changes in user requests, leading to better conversation results without extra tools at run time.
Open → 2610.33766v1

Tool calling speed improved by managing live user memory stores

SPECMEM: Accelerating Agentic Tool Calling via Live Memory Management

Abstract: Tool-calling agents are mostly optimized for accuracy; in deployment, however, their growing cost is latency: every tool call is decoded autoregressively. Speculative decoding accelerates this without changing the output, and retrieved past calls make strong drafts under real use, where each user's behavior tends to repeat. Yet existing systems keep no cross-session memory, or build the draft store once and never update it. We propose SpecMem, a tool-call memory kept alive and managed throughout deployment: verified calls are written back online into bounded per-user stores with LRU eviction, and a side-effect gate defers irreversible calls until verification confirms. On multi-session, multi-user query streams served by three open-weight models, a frozen store steadily loses acceptance while SpecMem holds it: a 27% gap in mean accepted tokens (+11% to +34% across benchmarks, tracking how much novel work arrives after the store is built) and up to 1.74x wall-clock speedup. Ablations attribute the gain almost entirely to keeping the store live; local per-user scoping adds no acceptance but closes a measured cross-user leakage channel, and gating on declared side effects, rather than on retrieval confidence, is what eliminates irreversible mistakes.

Thu 22 OctComputation and Language
The gist
Calling tools through AI agents usually takes time because each call is processed step-by-step. The authors propose SpecMem, a method that keeps track of past verified tool calls for each user and updates this memory continuously. This live memory helps the system guess and speed up future calls without making errors. SpecMem also carefully checks before making irreversible tool calls, avoiding mistakes. Tests showed this approach made tool calls faster and more accurate compared to previous methods.
Open → 2610.31840v1

Language models identified by unique one-word answers to questions

Forty Questions: Identifying a Language Model from One-Word Answers

Abstract: A model behind an API can be identified from its one-word answers to everyday questions such as "Name a sound" or "Name a poet". A model's profile records how often it gives each answer to each question. We attribute new answers to the model most likely to have given them. On a panel of 94 models, 40 answers identify the exact model 92.5% of the time, against 1.1% by chance. Profiles built through OpenRouter identified all ten models served by DeepInfra under blinded labels. A system prompt lowers accuracy by 6 to 16 points. We applied the method to 96 endpoints of 13 open models on OpenRouter, and 92 identify as the model they claim. Of the other four, one matches the claimed model with reasoning turned off. One serves Gemma 4 under the name Llama 4 Maverick: it names itself Gemma 4, and it is billed Gemma 4's token counts on every message. Two remain unresolved. Billing records also show context added to first-party models. Grok 4.7 is billed 1,247 input tokens for a one-line question that costs GPT-4o only 12 tokens, and five models state the current date without being billed for it. The questions come from a public census of model defaults, and the method costs cents per endpoint.

Tue 20 OctComputation and Language
The gist
It is possible to tell which language model answered a question by looking at just one-word responses to simple everyday questions. The authors show that by collecting patterns of these answers from many models, they can guess the right model over 90% of the time. They tested this on a large number of models through APIs and found some mismatches and hidden details like extra billing tokens added by certain services. This method is low-cost and relies on answers from common questions, making it practical to identify models behind APIs.
Open → 2610.28281v1

SteerSearch improves interactive search with mid-search user redirection

SteerSearch: Interactive Search Reasoning with Mid-Response Steering

Abstract: Search-augmented language models increasingly serve as long-horizon research agents, yet most open training methods assume a fixed user goal throughout search. We study mid-response steering, where users redirect a model during ongoing reasoning or retrieval. We identify answer-target anchoring, a failure mode in which fixed-query search agents retrieve evidence for the updated request but still answer an earlier question. We introduce SteerSearch, an RL framework that trains on interactive search trajectories with injected interventions to enable context-preserving target reassignment, and SteerSearch-Bench, a benchmark adapted from multi-hop QA datasets. We also investigate Contrastive Context Reward (CCR), an auxiliary signal for aligning post-intervention reasoning with the updated goal. Without CCR, SteerSearch improves average steering accuracy over Search-R1 from 11.1% to 31.3% on Qwen2.5-3B and from 36.6% to 44.6% on Qwen3-4B, while preserving standard single-query performance. Gains also extend to larger Qwen2.5 models and natural multi-turn conversations.

Fri 16 OctArtificial Intelligence
The gist
When people search for answers using AI models, they sometimes change their questions while the AI is still thinking. The researchers noticed that many models fail to update their answers properly after such changes. They created SteerSearch, a new method that helps the AI adjust its reasoning during the search to match the updated question. Their experiments showed that SteerSearch significantly improves the accuracy of such real-time user corrections without losing the ability to answer normal questions.
Open → 2610.24287v1

Tool-using agents learn when to act or ask for help

Agents' Last Doubt: Knowing When to Act and When to Step Back in Tool-Using Agents

Abstract: Tool-using LLM agents repeatedly face decision points under uncertainty, choosing between autonomous action, additional information gathering, and human escalation. These decisions require balancing task success against interaction cost and the risk of irreversible errors. Learning to make these trade-offs is challenging because existing interaction trajectories record only the outcomes of actions taken, leaving the consequences of alternative actions unobserved. To address this gap, we construct a counterfactual dataset containing 3,115 action outcomes from 1,036 decision points in ToolSandbox. At each decision point, we execute each legal action and evaluate its downstream outcomes based on task completion, interaction cost, and irreversible errors. Using this dataset, we train SURE-Act, a controller that selects actions for a frozen backbone agent at inference time. Our experiments show that learning to rank alternative actions improves offline action-selection utility by 0.0309 over a regression-only baseline. We further show that adding SURE-Act enables end-to-end gains of up to 0.0575 in ToolSandbox's Overall Score with the Qwen3-4B agent. Moreover, SURE-Act outperforms adapted decision rules from SAGE-Agent and Calibrate-Then-Act by 0.1506 and 0.1438, respectively. This work establishes the first counterfactual dataset for ToolSandbox and provides actionable insights into decision-making under uncertainty for tool-using agents.

Fri 16 OctArtificial Intelligence
The gist
Tool-using AI agents often face tricky choices about whether to act on their own, gather more information, or ask a human for help. The authors created a special dataset that records outcomes not only for actions taken but also for all possible alternative actions to understand these choices better. They used this data to train a controller called SURE-Act that helps agents decide the best move in uncertain situations. Their tests show this approach improves the agent’s performance and beats other decision-making methods. This work helps AI agents make smarter decisions when handling tools to avoid mistakes or unnecessary delays.
Open → 2610.24194v1

Model routing improves cost and performance in stateful AI agents

Whose turn is it? Context-guided model routing for stateful agents

Abstract: Model routing can reduce the cost of LLM applications by assigning simpler requests to cheaper models without sacrificing the quality of the response. However, most routers treat requests as independent. Stateful agents do not operate under this assumption: a user turn may require multiple model and tool calls, its difficulty depends on the state produced by earlier turns, and changing models can sacrifice reuse of the conversational prompt cache. We study zero-shot, cost-aware routing at the user-turn boundary, systematically varying the routing rule, access to session history, candidate models that differ in size, reasoning effort, or model provider, and prompt-cache behavior. We evaluate on multi-turn versions of SheetCopilot and SpreadsheetBench 2, where persistent workbooks and cross-turn dependencies make isolated routing decisions especially brittle. At a matched offloading rate, giving the router session history improves task pass rate over the same router without history, with gains for every model pair; history-aware routing remains within one point of always using the expensive model. We also found that cost depends on execution dynamics rather than token prices alone: the cheaper model's per-task cost advantage narrows from 6.8x to 3.2x on longer sessions. A closed form cache analysis further shows that downward switches (expensive to cheap) recover their cache cost within two LLM calls, whereas upward switches do not break even under the evaluated conditions. Together, these results show that agent routing is a sequential decision problem: effective policies must account for what happened earlier in the session and for the asymmetric cost of switching models.

Tue 13 OctArtificial Intelligence
The gist
Handling conversations with AI agents that remember past interactions is tricky because they often need to switch between simpler and complex models as the conversation progresses. The authors studied ways to decide which model to use for each user turn based on the conversation's history and model switching costs. They found that considering previous conversation turns helps maintain accuracy while reducing costs. Also, changing from a complex model to a simpler one is less expensive than going the other way, impacting how systems should manage these switches.
Open → 2610.19292v1

Agentic self-awareness improves AI task decision making

Agentic Self-Awareness: Aligning Knowing, Acting, and Holding

Abstract: Frontier AI agents increasingly act in environments where seemingly solvable tasks may lack prerequisites, contain conflicting requirements, or reveal blockers only during execution. We formulate agentic self-awareness (ASA) as the alignment of task-grounded feasibility judgments (know), warranted behavior (act), and responses to reconsideration (hold), distinguishing what a task permits from what an agent can solve. We build paired feasible and blocked tasks in Terminal-Bench 2.1 and retail $τ^2$-bench and introduce an authorship-aware M0-M3 rubric that separates recognizing a blocker from taking the certified deferral action for the correct reason. Separate probes assess stated feasibility beliefs and responses to a single challenge. In a Terminal-Bench protocol baseline, blocker recognition reaches 84.4%, but strict self-awareness (SA) reaches only 40.6%. Harness search without model-weight updates increases strict SA from 45.3% to 93.8% on reused Terminal-Bench search tasks; selection for abstention alone, however, can reduce success on feasible tasks. Independently initialized joint-objective searches improve both blocked-task SA and feasible-task solve on their own search sets. Retail held-out gains remain limited, and Terminal-Bench blocked-task transfer is not measured. This evaluation pipeline makes the recognition-action gap measurable and sets a two-sided target for improving agent decisions under changing task evidence.

Tue 13 OctArtificial Intelligence
The gist
AI agents often face tasks that seem solvable but can have hidden problems or blockers that show up only when trying to complete them. The authors describe "agentic self-awareness" (ASA) as the ability of an AI to know when a task is feasible, act accordingly, and reconsider its actions when new obstacles are found. They created tests to measure how well AI recognizes blockers and chooses correct responses, finding that many agents detect problems but fewer take the right action in response. This work introduces a system to better measure and improve AI decisions when faced with uncertain or changing tasks.
Open → 2610.19226v1

Large language models tested for compliance with human rights standards

UDHRBench: Benchmarking Human Rights in Large Language Models

Abstract: Large language models are increasingly deployed in contexts requiring adherence to ethical and legal norms, yet their alignment with internationally recognized human rights standards remains underexplored. We introduce UDHRBench, a large-scale benchmark for operationalising the evaluation of large language model compliance with the Universal Declaration of Human Rights. The dataset comprises 1,124 human rights confronting questions covering all 30 articles and 4,496 responses from four large language models (ChatGPT, Grok, Gemini, and Fanar), manually annotated by experts for alignment, with high inter-annotator agreement ($κ= 0.801$). Our analysis reveals systematic and non-trivial misalignments, showing that violations typically arise through seemingly rational reasoning: appeals to efficiency, risk management, or scientific neutrality, rather than explicit rejection of rights. We further propose an LLM-as-a-Judge framework for automated evaluation using zero-shot and few-shot prompting, achieving 89.8\% weighted F1-score in detecting human-rights misalignment, alongside a novel Alignment Leniency Score that quantifies how judge models resolve ambiguous cases. The dataset, annotations, and code will be released as open-source resources (https://sina.birzeit.edu/UDHRBench).

Sat 10 OctComputation and LanguageArtificial Intelligence
The gist
Language models like ChatGPT are used in many important areas, but how well they respect human rights is not well known. The authors created UDHRBench, a big test made of questions covering all parts of the Universal Declaration of Human Rights, to check how different models respond. Experts reviewed thousands of answers and found that models often break rights unintentionally by using logical but misaligned reasons like efficiency or risk. They also designed a way to have language models judge other models' answers automatically, with strong accuracy.
Open → 2610.15179v1

LLM agents improve task handling by refining their own execution setups

Harness^2: Recursive Agent Harnessing for an Open World

Abstract: LLM agents' capabilities depend not only on their underlying models but also on their execution harnesses. Recursive self-improvement (RSI) offers a promising approach to automatically evolve these harnesses. However, deploying RSI in open-world settings still faces two critical challenges. First, open-world tasks are typically unseen during the harness design phase and novel at test time, leaving no ground-truth verifiers to reliably guide online harness edits. Second, the RSI process remains highly task-specific and fails to generalize to novel scenarios. We introduce Harness^2, a plug-and-play, recursive agent harnessing framework that addresses these challenges through two connected levels of recursion. Task-level recursion refines the current task's harness using a contrastive Propose-Probe-Compose step that reliably guides edits without ground-truth verifiers, and can be scaled further via parallel or sequential recursion. Harnessing-level recursion then extracts experience from these edits to update a continual improvement harness, enabling the agent to better adapt to future tasks. Across three professional benchmarks and four model-harness configurations, a single step consistently improves GLM-5 and Gemini 3.5 Flash over their base harnesses, yielding gains of up to 11.5 points on WorkBuddy-Bench, 4.5 points on JobBench, and 11.8 percentage points on LAB, with further gains as the task-level recursion budget increases.

Fri 9 OctArtificial IntelligenceComputation and LanguageMachine Learning
The gist
Large language model (LLM) agents rely on how they are set up to perform tasks, not just on the model itself. The paper introduces Harness², a framework that lets these agents improve their task-handling methods by repeatedly editing and learning from their own setup without needing outside feedback. This recursive approach helps them adapt better to new tasks they haven't seen before. The authors tested this system on several benchmarks and found consistent improvements in performance.
Open → 2610.14224v1

MoF method personalizes black box AI models efficiently

MoF: Preference-Aware Mixture Modeling for Black-Box LLM Personalization

Abstract: Proprietary Large Language Models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks, yet aligning their outputs with diverse user preferences remains challenging. Existing personalization approaches for black-box LLMs often rely on user-specific scoring heads, causing the number of personalized parameters to grow linearly with the number of users and requiring additional adaptation for unseen users. To address these limitations, we propose Mixture-of-Facets (MoF), a scalable personalization framework for black-box LLMs that models user preferences as compositions of shared latent preference facets rather than dedicated user-specific parameters. MoF performs personalization through history-conditioned routing over shared facet heads, enabling personalization for users unseen during training without additional parameter updates. Across diverse personalization tasks, MoF delivers stronger personalization performance while maintaining a more scalable and parameter-efficient design than prior approaches. Additional analysis indicates strong generalization to unseen users.

Tue 6 OctArtificial Intelligence
The gist
It is hard to make large AI language models match what different people want because each user can have unique preferences. The authors introduce a way to personalize these AI models without needing separate custom settings for each user, which usually gets very complicated. Their method, called Mixture-of-Facets (MoF), combines shared pieces that represent user preferences, allowing it to work well even for new users it has never seen before. This approach makes personalization more scalable and uses fewer extra parameters than older methods.
Open → 2610.08330v1

Llm workflows cluster queries to improve task solving efficiency

RA-MoWE: Workflow-Affinity Embeddings for Query Clustering and Agentic Workflow Generation

Abstract: Agentic workflows enable large language models (LLMs) to solve complex tasks by coordinating reasoning, tool use, and verification. However, a workflow optimized for an entire task collection can overlook differences in the reasoning strategies that individual queries need, while searching for a new workflow for every query repeats costly optimization. To address this tradeoff, we introduce RA-MoWE, a framework that uses workflow-affinity embeddings to cluster queries and guide the generation of reusable expert workflows. Each embedding records how well a fixed set of reference workflows solves a query, revealing similarities in which reasoning strategies are effective. RA-MoWE uses each cluster's queries and average embedding to initialize and refine a specialized workflow through execution feedback. An embedding encoder predicts these embeddings from query text, allowing new queries to select a generated expert without first executing the reference workflows. On a 300-query test set drawn from four benchmarks spanning mathematics, science, and programming, RA-MoWE improves average task score by 4.04 percentage points over selecting among the reference workflows, while using 27.7% fewer language-model calls at inference.

Tue 6 OctArtificial Intelligence
The gist
Solving complex problems often requires a series of steps or a 'workflow.' The authors found that one general workflow doesn’t work well for all types of questions. They created a method called RA-MoWE that groups similar questions together based on which existing workflows work well on them. This grouping helps generate special workflows for each group, making it easier and faster to solve new questions without starting from scratch each time. Their tests showed this method improves accuracy while using fewer resources.
Open → 2610.07851v1

AgentMemGate stops false memories in conversational AI assistants

AgentMemGate: Addressing Speculation Contamination in Conversational Assistant Memory

Abstract: Conversational AI assistants with long-term memory extract facts from user messages into a store consulted in later conversations. A stated plan can enter that store as fact: a user who might move to Seattle may be recorded as already living there. We call this speculation contamination. Final-state memory benchmarks miss this error because they do not probe intermediate state and include few unresolved speculations. We present AgentMemGate, a write-time gate for profile-store memory that classifies extracted statements as speculation, completed event, correction, or other. Speculations remain outside memory, with conditions governing later promotion or deletion. We also contribute a dataset of multi-session conversations in which plans are confirmed, abandoned, or left unresolved. On our 147-conversation held-out set, Mem0 and Graphiti assert unresolved plans as current state for 35.2% and 27.3% of pending plans. On the core benchmark, AgentMemGate eliminates all observed contamination relative to the identical ungated pipeline (87.5% to zero for the most exposed extraction style) and raises task accuracy from 65% to 95%. On the harder held-out set, gated contamination is 3.4% to 5.7% and task accuracy rises by 9 to 13 percentage points. Our analysis identifies field matching as the main remaining bottleneck: realistic speculations often match no profile field and never reach the gate. We release our datasets, prompts, and evaluation code.

Tue 6 OctArtificial Intelligence
The gist
Conversational AI assistants often remember things users say, but sometimes they record plans or guesses as if they are facts, which can cause confusion later. The authors call this problem speculation contamination. They created AgentMemGate, a system that decides when a statement is just a plan or speculation and stops it from being incorrectly remembered as fact. Their method improved accuracy a lot and reduced errors in remembering user information during conversations.
Open → 2610.07707v1

Coding ai learns to ask right questions under budget constraints

Pareto-Dominant Clarification: Post-Training Coding LLMs via PPO-Lagrangian Budget Constraints

Abstract: Coding agents operating under ambiguous instructions or user prompts must decide whether to ask clarifying questions or attempt a solution directly. While clarification from the user may improve the correctness of the agent's solution, each back-and-forth interaction incurs user and system costs, forming an explicit accuracy vs. efficiency tradeoff. Existing works study clarification behavior but do not train policies under enforceable clarification budgets; penalty-based approaches typically require separate coefficient tuning swept across all clarification budget levels. We formulate clarification as a Constrained Markov Decision Process (CMDP) and post-train Qwen2.5-Coder-7B-Instruct with PPO-Lagrangian to optimize coding accuracy, subject to an expected question-budget constraint. Evaluated on HumanEvalComm with a GPT-4o-mini oracle simulator, the resulting policies reveal that untuned clarification behavior is Pareto-inefficient: budget-constrained policies can simultaneously achieve higher accuracy and lower clarification rates than the baseline model. Across budget levels, we observe a log-shaped Pareto frontier with diminishing returns to additional clarification. Gains arise not from simply asking more questions overall, but from improved question targeting and better code generation under ambiguity. Without explicit supervision, trained policies learn to allocate clarification budget non-uniformly, asking more frequently on tougher (multi-degradation) tasks. These results suggest that unconstrained interactive LLM systems may systematically use clarification inefficiently.

Fri 2 OctMachine Learning
The gist
When coding AI assistants get unclear instructions, they need to decide if they should ask questions or just try solving the task. The authors studied how to teach AI to ask just enough helpful questions without wasting time or irritating users. They trained a model to balance accuracy and the number of questions, finding that smart question-asking improves results more than simply asking more. Their approach showed that AI can learn to ask fewer questions on easy tasks and more on difficult ones, making interactions more efficient.
Open → 2610.04089v1

Large language models share most user personalization with tiny personal code

Does Every User Need a Private LoRA? Decoupling Personalization from Per-User Adaptation

Abstract: Personalized large language models often require a complete adaptation state for each user. However, this paradigm scales poorly as the user population grows. We revisit this design through the lens of personalization capacity allocation: how much adaptation capacity can be shared across users, how the shared capacity should be composed, and how much must remain user-specific. We answer them through three complementary empirical analyses. We find that independent user adapters contain substantial cross-user reusable structure, that the utility of reusable directions reflects both user relevance and variation across queries, and that user histories provide transferable signals for compact individual correction. Motivated by these findings, we propose LINEUP. It learns a bank of reusable low-rank personalization factors, composes them through user-conditioned recall and query-dependent calibration, and restricts target-user adaptation to a tiny user code over a shared correction space. This design decouples expressive personalization capacity from per-user trainable state. Each target user optimizes only eight scalars, while all shared components remain fixed. By comparison, the evaluated private-LoRA configuration uses 4.19 million per-user parameters. Our theoretical analysis gives a finite-step, finite-history risk bound and sufficient conditions for user-code refinement to improve on history initialization. Across six tasks spanning personalized classification, prediction, and generation, LINEUP leads on all 12 metrics, each averaged over three independent runs (e.g., reducing LaMP-3 RMSE by 11.4% relative to the strongest baseline). It maintains advantages under limited history. These results show that rich personalization can be supported primarily by reusable, conditionally composed shared capacity, while independent user adaptation remains confined to a tiny correction state.

Thu 1 OctMachine LearningComputation and Language
The gist
Personalized language models usually need a large set of parameters for each user, which gets harder to manage as more users join. The authors studied how much of the adaptation to users can be shared instead of unique for each person. They found that most personalization can come from shared factors reused for all users, while each user only needs to adjust a tiny amount of information. Their method, called LINEUP, lets each user optimize just eight small values instead of millions, yet still achieves better results across various tasks.
Open → 2610.02353v1

Generative flow networks create diverse expert conversations for training ai

Beyond Mode Collapse: Generating Diverse Synthetic Expert Conversations via Generative Flow Networks

Abstract: High quality synthetic data is central to post training LLMs for adaptive AI applications that represent the diverse expert strategies and decisions in conversations. Prompting LLMs directly or conditioning them on end use scenarios yields low diversity data that collapses onto dominant modes. We propose a method to generate diverse high quality synthetic data using Generative Flow Networks (GFlowNets). We show that training GFlowNets to generate latent conversation structure using a Gaussian mixture density over key interaction features (e.g., confusion episode dynamics, scaffolding directive balance) enables sampling expert strategies in proportion to their prevalence in the training data. Across two structurally distinct domains, tutoring and emotional support dialogues, our GFlow based synthetic data generation approach offers a better balance of fidelity, mode coverage and authenticity than reinforcement-learning and end to end LLM baselines, without copying training data. Evaluated on three downstream outcome prediction tasks, classifiers trained on synthetic GFlowNet generated conversations provide a stronger training signal than competitive synthesis baselines.

Tue 29 SeptArtificial IntelligenceComputation and LanguageMachine Learning
The gist
Generating varied and realistic expert conversations is hard because usual AI methods tend to repeat common patterns. The authors used a new approach called Generative Flow Networks to create diverse synthetic dialogues by modeling key features of conversations. This method better captures different expert strategies and produces data that helps train AI models more effectively. They tested their method on tutoring and emotional support chats and showed it outperforms other popular techniques.
Open → 2609.38359v1

Chat systems let users edit conversation history to reduce confusion

Mutable Transcripts: Mitigating Context Pollution through Editable Conversation State

Abstract: Contemporary large language model (LLM) chat systems treat conversation history as an immutable sequence of turns that defines the model's working context. However, user intent in real interactions is not static: it evolves through correction, refinement, and shifting constraints. This mismatch between dynamic intent and static transcripts can result in context pollution, where outdated or irrelevant information persists and continues to influence subsequent responses. We introduce mutable transcripts, a new interaction paradigm that enables users to revise prior turns through natural language edit requests, allowing the conversation history itself to be updated rather than appended. This reframes the transcript from a passive record into an editable representation of conversational state. We present a working prototype that integrates transcript-level revision into a standard chat interface and evaluate its feasibility through a controlled user study (n=17) and an illustrative transcript analysis of representative interaction scenarios. Participants significantly preferred mutable transcripts over standard chat across measures of clarity, confidence, and ease of use, with reduced intent to restart conversations. Transcript analysis of representative user study conversations shows that mutable transcripts can reduce conversation length and eliminate obsolete retained context. These findings provide initial evidence that user-driven revision of conversational history can improve interaction quality and help maintain a more current representation of user intent. The source code and prototype can be accessed at https://github.com/QxLabIreland/ReChat

Fri 25 SeptArtificial Intelligence
The gist
In normal conversations with AI chat systems, past messages stay the same even if your thoughts change. This can cause confusion because the AI keeps remembering old, no longer relevant details. The authors created a system where people can edit previous parts of the conversation to fix or update what they said. This helps the AI better understand what you really want and makes chats clearer and shorter. Users found this editable chat history easier and more confident to use.
Open → 2609.31354v1

Better deciding when to revise answers improves question accuracy

Return or Revise? Learning When Revision Helps Retrieval-Augmented QA

Abstract: We consider the decision of whether to return an existing draft answer or revise it using retrieved evidence, as in answer-revision systems. Draft confidence estimates whether the current answer is correct, but the decision requires estimating the effect of a specified revision. For offline training and evaluation, we grade both the returned draft and its candidate revision under the same correctness judge, which makes repair, harm, and the gap to an oracle observable. We call this paired effect its recoverability, and we train policies to predict it before revision. On 25,870 held-out open-domain questions across three revision setups, a scorer trained on the paired outcome has greater area under the accuracy--revision-rate curve than a matched draft-correctness scorer in all nine Llama setup--seed fits, and gains 0.23--0.68 accuracy points on average at development-selected thresholds, a difference significant across training runs only for dense retrieval. The resulting policy improves on always revising and on average closes more than a third of the oracle gap, although it still applies 38--46% of the harmful revisions. When a draft-free standard-RAG answer is also available, however, choosing between the draft and that answer is stronger by about two points for Llama and four for OLMo, and adding candidate revision as a third option yields no significant gain. Recoverability describes one revision; its value as an available action also depends on the alternatives.

Thu 24 SeptComputation and LanguageInformation RetrievalMachine Learning
The gist
When AI systems answer questions by first making a draft response and then retrieving extra information to revise it, it can be tricky to know if revising will actually help. The authors studied how to predict whether revising an answer will improve it or cause harm. They trained a system to estimate this 'recoverability' before attempting revisions, which led to more accurate answers than always revising or just trusting the original draft. However, sometimes harmful revisions still occur, and choosing from multiple answer options can work better in some setups.
Open → 2609.30087v1

Structured semantic supervision improves product search relevance

Beyond Relevance: Structured Semantic Supervision for Product Search with LLM-Augmented Annotations

Abstract: E-commerce search requires distinguishing products that are merely related to a query from those that directly satisfy the user's shopping intent. We augment query-product pairs with structured LLM-generated query and product attributes and human-validated relevance, explanations, and centrality judgments, and evaluate these signals using a simple dual-encoder retriever and MLP re-ranker. On an augmented subset of ESCI, a human-feature oracle reaches $0.9382$ nDCG@10, while a human-free trained $Q+P$ configuration reaches $0.9258$. Synthetic approximations of the human signals reach $0.9150$ overall but provide substantial gains for difficult, low-performing queries. Ablations show that most of the oracle improvement comes from post-edited explanations and annotator comments rather than the scalar centrality feature, suggesting that LLMs are most useful for exposing and approximating structured semantic supervision rather than replacing human judgment directly.

Sun 20 SeptInformation RetrievalComputation and Language
The gist
Finding the exact product a shopper wants online is hard because many products are only somewhat related to what they type. This paper shows that adding detailed, structured information generated by large language models (LLMs) about queries and products helps search systems understand what people really want. The researchers combined these AI-generated signals with human checks and found their method made product search more accurate. They also found that human-edited explanations added the most value, meaning the AI helps show useful details rather than replace humans completely.
Open → 2609.23646v1

JustMem improves memory use for better long conversations

JustMem: Just-Enough Memory Access for Long-Term Conversations

Abstract: Efficient long-term conversational memory requires retrieving sufficient evidence without indiscriminately expanding the context presented to the language model. This is challenging because relevant evidence may be distributed across multiple sessions, while compression may discard details needed for answering. Different queries therefore require different forms of memory access. To capture these demands, we formulate memory access along two dimensions: discovery breadth, which controls how broadly evidence is searched, and reading fidelity, which controls whether evidence is read in compact form or recovered from the original conversation. Based on this formulation, we introduce JustMem, which stores conversation history as compact atomic memories and adapts memory access along these two dimensions to each query. Specifically, LOOKUP handles local evidence, COMPOSE broadens discovery for distributed evidence, and REPLAY increases reading fidelity for fidelity-sensitive evidence. On LoCoMo and LongMemEval-S, JustMem achieves the highest mean accuracy and retrieval recall among the compared memory systems while using substantially fewer generative-model tokens for memory construction and inference.

Thu 17 SeptComputation and Language
The gist
Handling long conversations can be tricky because important details might be spread out over many sessions and some memory-saving methods lose useful information. The authors propose JustMem, which stores conversation history in small pieces and chooses how it looks up information based on the question being asked. It can search narrowly, broadly, or recall exact details as needed. Tests show JustMem is more accurate and efficient than other memory systems for long conversations.
Open → 2609.19877v1

BenchCompass reveals challenges for language models in payments tasks

BENCHCOMPASS: From Scores to Signals for Training and Harness Decisions in Payment-Domain LLMs

Abstract: Payment operations are a critical financial infrastructure, but the value of large language models in this domain remains unclear because payment rules change quickly, evidence is fragmented, and decisions depend on transaction state, participant role, region, and payment rail. Existing benchmarks do not isolate whether failures come from missing payment-rule knowledge, poor use of supplied evidence, or brittleness under imperfect harness inputs. We introduce BENCHCOMPASS, a payment-domain benchmark whose construction pipeline builds scenario-grounded tasks from typed evidence packs, applies LLM-based quality checks, creates task-input attack variants, and reserves final item admission for domain experts. The release contains an expert-reviewed Pro benchmark covering payment knowledge, context-grounded scenario reasoning, and Attacked Open robustness, plus a lower-assurance Normal pool for inspection and future curation. Across 16 model variants, BENCHCOMPASS shows qualitatively different failure modes: missing parametric payment knowledge, incomplete reasoning over supplied rules, and failure to reject plausible but invalid workflows. The benchmark remains unsaturated: the best frontier model reaches 89.6% on Open Context-Grounded Reasoning and 81.7% under attacked inputs, while a representative 32B open-weight model reaches 69.8% and 42.6%. Benchmark data and code are available at https://github.com/ant-intl/BenchCompass.

Wed 16 SeptArtificial Intelligence
The gist
Payment systems are complicated and change often, making it hard for AI language models to help with payment decisions. The authors created BenchCompass, a test set that checks if models understand payment rules, use evidence well, and handle tricky situations. They found that current models still struggle with missing knowledge, reasoning, and rejecting wrong payment steps. Their work provides a way to measure AI progress in this area.
Open → 2609.18270v1

Structured text improves table reading and question answering tasks

Tables Decoded: DELTA for Structure, TARQA for Understanding

Abstract: Table understanding is a core task in document intelligence, encompassing two key subtasks: table reconstruction and table visual question answering (TabVQA). While recent approaches predominantly rely on vision- language models (VLMs) operating on table images, we propose a more scalable and effective alternative based on structured textual representations. These representations are easier to process, align more naturally with LLMs, and eliminate the need for language-specific visual encoders, making them particularly suitable for multilingual documents. We present DELTA, which separates physical structure recognition, logical structure recognition, and OCR to extract both layout and content accurately. DELTA outputs tables in Optimised Table Structure Language (OTSL), a compact and unified format that encodes cell arrangements and textual content. On table structure recognition (TSR), DELTA achieves TEDS- Structure scores comparable with state-of-the-art methods across FinTabNet, PubTabNet, and PubTables-1M. We further establish its robustness on non-English tables through our curated Hindi benchmark, TORQUE. Building on this, we introduce TARQA, an LLM fine-tuned on OTSL sequences. Our approach yields gains of 9.3 p.p. on WTQ (TabQA) and 9.2 p.p. on FinTabNetQA (TabVQA), respectively. On TORQUE, our method ranks second among all VLMs and DELTA + LLM variants. We release our code, models, and benchmark at: https://github.com/Tihiitborg/Tables-Decoded

Tue 15 SeptComputer Vision and Pattern RecognitionMachine Learning
The gist
Understanding tables in documents is important but tricky, especially when tables are in images and different languages. The authors developed DELTA, a way to turn tables into a clear written format that captures both layout and content accurately. They also created TARQA, a tool that uses large language models to answer questions about these tables using that format. Their new approach works well across several datasets and languages, including Hindi, and performs better than many previous methods.
Open → 2609.17458v1

CoArena ranks computer-use agents with real-time user tasks

CoArena: Evaluating Computer-Use and Multi-Agent Systems in Real Time

Abstract: Static benchmarks for computer-use agents fix a task set at release and score every system against it once. That makes them reproducible, and it lets them drift from what they should measure: a fixed task set ages, leaks into training corpora, and cannot follow how people actually use agents from week to week. CoArena measures use directly. Real users submit tasks; two systems, each a single model or a multi-agent pipeline behind the same tool interface, execute the same task concurrently in identical sandboxed desktops; users judge the two outcomes without knowing which system produced them; and a public leaderboard is refit from those judgments. The central contribution is a formal account of what makes such an evaluation real-time. We define real-time as five measurable properties, each with an equation and a worked example: continuous task arrival, live concurrent execution, online rating updates, freshness with contamination resistance, and bounded feedback latency from a failed run to a reusable training environment. The rating methodology follows in full: the Bradley-Terry pairwise model, its likelihood with weighted observations and ties, the penalized maximum-likelihood estimator, and the streaming update applied when a single vote arrives (a stochastic-gradient step on the same likelihood, recovering Elo). It gives confidence intervals from the observed information and a cluster-robust sandwich, rank bands from a parametric bootstrap, the rule by which a new system enters the board, and the convergence rate of the estimate. Vote quality is treated with inter-judge agreement statistics, redundant judging, and explicit handling of ties and abstentions. A five-system example with 211 votes is carried from the vote matrix to ratings, intervals, and rank bands. Every number is derived from stated inputs or labeled illustrative; none is a measurement of a deployed system.

Sun 13 SeptMachine Learning
The gist
Current tests for computer-use agents often use fixed task sets that can become outdated and don't reflect how people use these agents over time. The authors propose CoArena, a system where real users send tasks to two competing agents running side-by-side, and the users judge which agent performs better without bias. This approach updates rankings continuously based on live feedback and uses a rigorous statistical model to keep the rankings accurate over time. The paper also details how such real-time evaluation can be mathematically defined and implemented.
Open → 2609.14239v1

Memory system improves long-term knowledge retention in AI agents

LifeFuse-Mem: Lifecycle-Aware State Fusion Against Temporary Overwriting for Long-Term Memory

Abstract: Long-running LLM agents require memory mechanisms that maintain coherent internal states across interactions. We study a lifecycle-labeled memory setting in which write episodes provide lifecycle metadata during training, and phase-aware readout is used during evaluation. This setting reflects the need to distinguish information that should remain influential across future interactions from information that should affect only the current context. A mismatch between these lifecycles can cause temporary information to overwrite durable knowledge, leading to behavioral drift in persistent agents. Within this setting, we introduce \textbf{LifeFuse-Mem}, a lifecycle-aware neural memory framework that separates information according to its temporal commitment. LifeFuse-Mem uses dedicated memory components and lifecycle-aware updates to allow stable and transient knowledge to evolve locally without converting temporary context into durable state. On the controlled anti-overwrite benchmark, LifeFuse-Mem improves acquisition-controlled retention and reduces temporary overwrite; on two public long-memory benchmarks, it remains broadly competitive. These results suggest that explicit lifecycle signals can help diagnose and mitigate overwrite in compact online memory.

Fri 11 SeptArtificial Intelligence
The gist
Some AI programs need to remember information over a long time, but sometimes temporary facts can accidentally erase important lasting knowledge. The authors studied a way to label memories with their expected lifespan during training and then read from them carefully during use. They created a system called LifeFuse-Mem that keeps temporary and permanent memories separate, so temporary info doesn’t overwrite important facts. Their tests show this method helps AI agents keep useful memories longer without confusion.
Open → 2609.12436v1