Papers for

enterprise automation teams

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Banking workflow agents face challenges completing tasks accurately and fully

Bank-World: A Benchmark for Evaluating Agents on Banking Workflows

Abstract: Bank-World is a benchmark of 50 banking tasks across two simulated institutions. Completing banking work requires more than producing the right calculation: agents must reconcile records, obtain permission, execute transactions, and keep instructions and status reports consistent as circumstances change. Bank-World tests these requirements through single-turn assignments and multi-turn interactions with simulated employees who supply information, approve actions, and revise instructions. Grading combines executable checks of recorded actions and state with model judgments of task requirements, including whether valid alternative solutions satisfy them. Across 1,600 attempts by four models, we find that correct decisions often fail to become completed work. Agents charge fees twice, distribute the correct total in incorrect amounts among the recipients, and execute payments while approval is pending. Their strengths also differ: Sol leads single-turn completion, Opus has the highest observed multi-turn completion rate, and Grok succeeds across more multi-turn tasks but less consistently. Bank-World makes these differences inspectable through task-level results and worked examples connecting source evidence, agent decisions, and simulated consequences. We retain the task environments, executable graders, and full trajectories privately to reduce contamination and preserve the benchmark for future evaluations.

Thu 29 OctArtificial Intelligence
The gist
Handling banking tasks is more than just getting the right numbers—it also requires keeping records straight, getting approvals, and adapting to changes. The authors created Bank-World, a set of 50 simulated banking tasks with two fake banks to test if AI agents can handle these complexities. They found that while AI can make some correct choices, they often mess up when finishing tasks, like charging fees twice or paying before approval. Different AI models had different strengths in how well and how consistently they managed the tasks.
Open → 2610.39733v1

Tool using language models risk unauthorized side effects in tasks

Task Success Is Not Enough: Side-effect-Aware Evaluation of Tool-Using Language Model Agents

Abstract: Tool-using language model agents are usually evaluated by whether they finish the user's task. That metric misses a common failure: an agent can complete the requested work while also taking an unauthorized side-effecting action, such as writing to the wrong object, expanding the scope of the request, or contacting an unintended recipient. We call this failure mode collateral damage (CD). We introduce SABench (Side-effect-Aware Benchmark), a benchmark of 201 email, calendar, and ticket tasks. Each task crosses one of four state structures (independent, dependent, externally reachable, cross-tool) with one of four trap families (ambiguity, mis-targeting, cascade harm, overreach), and each task has a deterministic trace-based oracle for both goal satisfaction and authorization compliance. Across 2,211 model-task pairs from 11 frontier LLMs spanning major commercial and open model families, 15.2% of all runs incur CD, with risk concentrated in ambiguity tasks (38.8% violation rate) while non-ambiguous traps average 8.4% CD. Among goal-satisfied runs, 12.8% still produce unauthorized writes. Cross-tool and dependent states have about twice the CD rate of independent states. CD rates vary by 4.7x across models at comparable goal rates, which suggests that task success alone is a poor proxy for safe tool use. The SABench task corpus is released at https://huggingface.co/datasets/Phaedrus/sabench; the traces, oracle, code, and analysis pipeline are released at https://github.com/hjj194/sabench-artifact.

Sun 25 OctArtificial IntelligenceComputation and Language
The gist
Language models that use tools to help complete tasks are usually judged successful if they finish the job. This paper shows that sometimes these models complete tasks but also make unintended or unauthorized changes, called collateral damage. The authors created a benchmark with many tasks involving email, calendars, and tickets to measure these side effects. They found that some types of tasks and states cause more of these errors, and different models vary greatly in this risk. Their findings suggest just finishing a task isn’t enough to say these models are safe.
Open → 2610.34167v1

Explicit responsibility boundaries improve safety in language model agents

Responsibility Boundaries for Agentic Systems: A Comparative Evaluation of Probabilistic Agentic Programming

Abstract: Language-model agents combine reasoning, tools, memory, delegation, and external effects. Authority is often distributed across prompts, host code, configuration, and logs. Probabilistic Agentic Programming (PAP) makes responsibility boundaries explicit programming abstractions connecting mandate, authority, effects, and evidence. We compare PAP with programming paradigms, agent-oriented and BDI systems, workflow and policy languages, and LLM tool-use frameworks along seven dimensions. A mediation kernel formalizes checks before tool execution, operation-bound approval, resource limits, and evidence. Inductive proofs establish conditional safety for arbitrary model responses, assuming correct guards and complete mediation. A backward simulation matches graph executions to source executions in a defined sequential Touba subset. Its human checkpoints release program continuations; operation-bound approval remains a separate kernel contract. Finite-state exploration checks the model. Scripted enforcement tests exercise selected refusal paths and permitted controls. An exploratory study combines seven business cases, including two three-agent workflows, seven models from three providers, five methods, and five repetitions per case and method. Shared English instructions and exact checks of stored outcomes yield 1,225 assessed observations. Native Touba completes all 35 observations on six models and 30 on the remaining model; structured Touba ranges from 21 to 35. Native matches the highest observed count for each model, including ties with the frameworks. All paired case intervals include zero. These observed counts describe the evaluated tasks; the runtime authority boundary remains fixed. The results connect responsibility contracts, execution rules, and evidence.

Fri 23 OctSoftware EngineeringArtificial IntelligenceProgramming Languages
The gist
Language model agents can do many things like reasoning and using tools, but it’s hard to clearly say who or what is responsible for their actions. The authors explain a way called Probabilistic Agentic Programming (PAP) that makes these responsibility boundaries clear and checks that agents act safely. They tested PAP on different business tasks with many models and found it can reliably enforce rules and evidence for agent actions. This helps keep AI agents accountable and within limits when they perform tasks.
Open → 2610.33312v1

Document formatting cues lost during text ingestion cause errors

What the Author Wrote, What the Agent Reads: Silent Cue Loss in Enterprise Document Ingestion

Abstract: Enterprise agents act on documents. Before an agent can follow a policy, answer a compliance question or file a ticket, some ingestion layer must turn a Word file or PDF into tokens - and that layer is where an agent deployed in the wild silently loses information. We show that one layer is lost by every mainstream converter - the meaning an author encodes in how text looks - and that the loss makes agents confidently wrong rather than visibly incomplete. We build a task from 76 real policy documents: recover the requirements the author singled out, scored against the file's own OOXML properties by a deterministic matcher with no LLM judge. Formatting-blind ingestion collapses from micro F1 0.690 to 0.150 (docling) and 0.075 (plain text), a gap that replicates across six answering models from two vendors - five Claude models spanning the Opus, Sonnet and Haiku tiers, and OpenAI's GPT-5.6. An agent that cannot see the cue guesses, and docling's false-positive rate on requirements the author did not flag is four times ours. A span-level leave-one-out localises the loss to two cues - highlight (+0.302) and colour (+0.262), both Holm-adjusted p < 1e-4 - while bold, italic, monospace and strikethrough each sit within +/-0.003 of zero. The same asymmetry appears in PDF. Colour is also the cue no mainstream converter preserves and no parsing benchmark scores, and it is common: 32.8% of 6,460 documents carry a deliberate colour or highlight, 10.3% on an obligation. A pre-registered test shows that typing a cue for the agent adds +0.085 beyond merely preserving it (Holm-adjusted p = 0.049). We validate the labels against rendered page images through an independent modality (kappa = 0.72) and bound what wording alone can achieve (AUC 0.664). For agent builders the practical finding is that ingestion, not reasoning, is where this reliability is won or lost, and that the fix is cheap and measurable.

Wed 21 OctComputation and LanguageArtificial IntelligenceInformation Retrieval
The gist
When computers read enterprise documents like Word or PDF files, they first convert them into plain text that agents can understand. This paper shows that during this conversion, important visual clues—especially color and highlighting used by the author to emphasize requirements—are lost. Losing these cues causes automated systems to make confident but wrong decisions. The authors found that preserving such formatting information significantly improves the accuracy of policy document understanding. They suggest that fixing the way documents are processed before reading is a cheap and effective way to improve reliability.
Open → 2610.30026v1

Computer agents struggle to detect and recover from errors in long tasks

CUA-Recovery: Benchmarking Error Recovery in Long-Horizon Computer-Use Agents

Abstract: As computer-use agents (CUAs) move to long-horizon workflows spanning multiple applications, undetected errors can silently corrupt persistent state and cause cascading failures in subsequent actions. Reliable error detection and recovery have become critical bottlenecks for building robust CUAs. However, existing benchmarks focus primarily on end-to-end task performance, providing limited insight into whether agents can detect and recover from errors during execution. The few benchmarks that evaluate recovery rely largely on manually injected errors, which may not reflect the types or horizons of failures induced by real agent policies. We introduce CUA-Recovery, a benchmark comprising test and training suites across multiple desktop applications. We construct it using ReRail, a framework for synthesizing long-horizon tasks and trajectories in a persistent desktop environment. ReRail composes MyPCBench tasks into new long-horizon workflows with executable ground truth, collects verifier-confirmed failures from multiple policies, and turns them into verified recovery trajectories. CUA-Recovery comprises a human-verified test suite of 2,250 erroneous states over 100 tasks and a training suite of 8.6K trajectories over 377 tasks. Evaluation on the test suite shows that agents struggle both to recognize inherited errors and to complete the task. Across six agents and six takeover depths, Error Awareness Rate (EAR) reaches at most 44.0% and Pass@3 at most 25.5%. After fine-tuning Qwen3.5-35B-A3B on the training suite, its average EAR rises from 19.9% to 35.2% on held-out recovery states, and its Rubric Score goes from 5.0% to 16.1%. It also gains 15.3 points on MyPCBench and 10.8 points on Odysseys for Pass@3, showing that the training trajectories help both with recovery and with completing tasks from a clean start. Our code is available at https://github.com/xu-hu-2002/CUA-Recovery.

Tue 20 OctArtificial IntelligenceComputation and LanguageMachine Learning
The gist
When software agents use computers over long periods and across many programs, small errors can sneak in and cause bigger problems later. The authors created a new test called CUA-Recovery to see how well these agents notice and fix errors during their work. They found that current agents often miss these mistakes and struggle to finish tasks correctly. Training agents on specially made examples improved their error awareness and task success rates, helping them recover better and do tasks from the beginning more successfully. This new benchmark helps evaluate and improve the reliability of computer-use agents over long workflows.
Open → 2610.27645v1

OpenSaC library improves deep research agents with programmable search

OpenSaC: A Python Library for Search as Code in Deep Research Agents

Abstract: Deep research agents investigate complex questions through iterative search, source examination, and evidence synthesis. A conventional workflow alternates reasoning with individual search and reading actions. Search as Code introduces a new paradigm: agents write programs that compose search, source reading, reranking, model calls, and local processing, returning selected observations for further reasoning. To support this workflow in existing coding agents, we present OpenSaC, an open-source Python library with a companion Search as Code skill. Configurable capability interfaces let agents combine these operations while retaining evidence for inspection and reuse. Compared with standard Codex using its built-in web search, the OpenSaC configuration uses less execution time and fewer model tokens on BrowseComp and WideSearch, with higher accuracy on BrowseComp and slightly lower row F1 on WideSearch. We aim to lower the barrier to adopting Search as Code and provide an open foundation for community development and transparent research on deep research agents.

Mon 19 OctArtificial IntelligenceInformation Retrieval
The gist
Sometimes AI agents try to answer difficult questions by searching and reading bits of information step by step. The authors propose a new way where these agents write small programs that combine searching, reading, and processing information automatically. They created OpenSaC, a free Python library that helps these agents work this way, making their searches faster and often more accurate. This library also keeps track of all evidence found during the search so it can be checked or reused later.
Open → 2610.25779v1

Computer use agents improve new software tasks with user manuals

Adaptation of Computer Use Agents to New Software with User Manuals

Abstract: Recent advances in multimodal foundation models have significantly improved the ability to operate real-world software through graphical user interfaces (GUIs) using visual observations and natural language instructions. However, maintaining strong performance on unseen applications remains challenging, as practical tasks often require long-horizon procedural knowledge and application-specific operation patterns. In addition, constructing executable tasks and collecting corresponding trajectories for training still heavily rely on human-annotated data, incurring substantial cost. In this paper, we propose MAGE (Manual-augmented computer use AGent Execution and learning), a framework that leverages user manuals as a scalable source of procedural knowledge to improve both inference and training. During execution, MAGE retrieves task-relevant manual content and uses a planner to generate explicit procedural guidance for CUAs. To further account for the discrepancies between manuals and actual GUI environments, such as version differences or user settings, MAGE also adapts the agent policy through environment interaction. During training, MAGE converts manuals into executable task-plan pairs and uses the plans to guide environment rollouts toward task-relevant states; the resulting trajectories are then used to optimize the policy without human-authored demonstrations. This unified design enables effective utilization of manuals for both task execution and training, reducing reliance on human-authored training data. Experimental results across six applications demonstrate improvements in task success rates over strong baselines, highlighting the effectiveness of manual-guided planning and policy adaptation in new software environments. We provide the code and reproduction scripts at https://github.com/ku-autoai/MAGE.

Fri 16 OctArtificial Intelligence
The gist
It can be hard for computer programs that control software by watching and clicking to work well with new or updated applications. The authors propose a new method called MAGE that uses user manuals to guide these computer agents on what to do, helping them understand complex tasks better. MAGE also learns from interacting with the software itself to handle differences between manuals and actual screens. This approach reduces the need for expensive human-created training data and works well across several different software applications.
Open → 2610.23012v1

Mentors improve task compliance of ai assistants using procedural manuals

MENTOR: Procedural Shielding of Tool-Using LLM Agents

Abstract: Tool-using LLM agents are increasingly used to automate workflows governed by procedural manuals, such as company rules for booking travel or procuring hardware. These manuals are written in natural language for humans and describe general requirements over such complex workflows. Ensuring compliance of LLM agents with these manuals is difficult, and state-of-the-art techniques use either LLM judges or manually written rules to forbid problematic tool-calls locally at runtime. This is insufficient because (i) only blocking tool calls keeps the agent clueless about how to make correct progress towards fulfilling its task, and (ii) can lead to overly conservative behavior of the agent, as tool calls might only be problematic in certain contexts. To overcome these challenges, we introduce MENTOR, which (i) additionally provides guidance for the agent to make procedural-compliant progress on the task, and (ii) ensures that this compliance is context-aware. We achieve these goals by building a novel history-dependent runtime shield that is obtained from the procedural manual via a multi-step autoformalization pipeline. At its core, MENTOR utilizes a game-based technique to ensure robustness against runtime uncertainty without over-resticting agents' behavior. This yields a formally assisted framework for procedural shielding of LLM agents that improves compliance well beyond existing methods. In particular, MENTOR lifts the compliance of shielded light-weight open-source LLMs above that of non-shielded proprietary models. Since a shield is synthesized once per manual and then serves every task, a MENTOR-shielded agent becomes a viable choice for repeatedly executed workflows, potentially reducing deployment costs when the offline construction cost is amortized over repeated tasks.

Mon 12 OctComputation and LanguageMachine LearningLogic in Computer Science
The gist
Many AI assistants use tools to help with tasks that follow complicated rules written for humans. Existing methods try to stop them from making mistakes but don’t guide them on how to do the right thing, leading to confusion or cautious behavior. The authors created MENTOR, which not only blocks wrong actions but also helps the AI understand how to follow the rules properly depending on the situation. They do this by converting human manuals into formal rules that guide the AI step-by-step, making it better at following procedures than current approaches. This is useful for tasks done over and over again, where building the guide once can save time and money later.
Open → 2610.17039v1

Large language models act and interact in the real world

Agentic Large Language Models

Abstract: Large language models (LLMs) have taken the world by storm. Their ability to communicate with us in our own language, and to reason about our world, has made them a favorite tool for writing, answering questions, and much more. Their answers already influence our decisions and our actions. What would happen if LLMs themselves could act in the world directly? What would happen if LLMs could interact with each other? Agentic LLMs combine the power of large language models with the ability to act in the real world. In this book we will discuss how agentic LLMs work, how they act, how they interact, and what this means for us, as users.

Fri 9 OctArtificial Intelligence
The gist
Large language models (LLMs) can already understand and generate human language, helping us with many tasks like writing or answering questions. This paper explores what happens when LLMs not only generate text but also take actions in the world and communicate with each other directly. The authors explain how these "agentic" LLMs work, the ways they can act autonomously, and how their interactions might affect users. They also discuss what it means for people who use these models and the potential for future developments.
Open → 2610.13455v1

System adapts collaboration by measuring ai competence on tasks

Competence-Aware Orchestration for LLM-Based Multi-Agent Systems

Abstract: LLM-based multi-agent systems (MAS) have emerged as a promising paradigm for enhancing complex reasoning through role specialization and collaboration, yet a key challenge remains: how can collaboration be adapted to a backbone's competence on the current query? This problem matters because more collaboration is not always better--richer coordination may help on difficult cases, but it also incurs additional token cost and redundant interactions. We propose a two-stage competence-aware framework for automatic MAS orchestration. It first models the LLM backbone's task-conditioned competence for the current query, and then uses this signal to adapt role activation and the communication graph. Across six benchmarks, our method achieves an average score of 93.81%. Experiments with six LLM backbones of varying competence levels further show that our method adjusts collaboration according to the inferred competence: for higher-competence backbones, it improves average accuracy from 87.22 to 89.97 while reducing token cost by 36.26%; for lower-competence backbones, it yields a further average gain of 4.58 points over existing methods. The same trend also holds on unseen backbones.

Fri 9 OctArtificial Intelligence
The gist
Different AI tools have strengths that vary depending on the question being asked. The authors introduce a way to figure out how good an AI is at a task before deciding how many helpers it should use and how they talk to each other. This helps make teamwork smarter, spending less effort when the AI is strong and adding more support when it's weaker. Their tests showed this approach improves accuracy while saving resources across many AI systems and tasks.
Open → 2610.13144v1

Workflowops improves multi agent task workflows using learned collaboration patterns

WorkflowOps: Learning Agent Collaboration Priors for Multi-Agent Workflow Orchestration

Abstract: Multi-agent systems are increasingly deployed for complex knowledge work, yet their orchestration layers remain largely memoryless: each new task is decomposed, assigned, and executed from scratch with no benefit from prior successful executions. We present WorkflowOps, a multi-agent workflow orchestration framework that learns agent collaboration priors from historical workflows and expands its agent pool on demand to cover new capability requirements. Our approach introduces three coupled mechanisms. First, a transition probability matrix captures pairwise agent collaboration frequencies from past workflows and applies them as soft guidance during DAG workflow construction through intra-layer ordering optimization, probability-thresholded edge suggestion, and transitive reduction for parallelism maximization. Second, a sufficiency-driven agent creation loop detects capability gaps via semantic matching scores, generates specialized agents through an LLM, and simultaneously injects them into the collaboration matrix, so that newly created agents are immediately usable with predicted collaboration priors. Third, a layered semantic matching strategy uses pre-trained sentence embeddings for fast, deterministic capability matching as a first pass, invoking LLM verification only for low-confidence cases, thereby reducing LLM routing calls by over 80\% compared to pure-LLM approaches. Experiments on mixed code, math, and question-answering suites show that WorkflowOps improves end-to-end pass rates over recent workflow-construction baselines, with the largest gains on structured, decomposable tasks where past agent handoff patterns transfer.

Tue 6 OctArtificial Intelligence
The gist
Coordinating multiple agents to work together on complex tasks is hard, and usually each new task starts without remembering past successes. The authors introduce WorkflowOps, a system that learns how agents have successfully collaborated before to guide new task planning. It also automatically creates new specialized agents when needed and uses smart matching to assign tasks efficiently while reducing reliance on large language model calls. Their experiments show that WorkflowOps improves the success rate in complex, structured tasks by reusing past collaboration patterns.
Open → 2610.07860v1

Agent skills mining improves task performance with domain tailored methods

Mining Agent Skills from Production Traces

Abstract: Agent skills that record procedural instructions are increasingly mined from execution traces rather than curated by hand. Skill-mining pipelines often use known task outcomes or feedback to guide skill construction. In production, reliable information on whether a run has succeeded may be unavailable. We study how the sampling of execution traces, access to success or failure information, and the form of the mined skills affect downstream task performance. Holding the mining pipeline fixed, we compare six combinations of mining evidence and skill forms. Mining evidence has three levels: successful trajectories only, successes and failures with their outcome labels, or the same mix with labels withheld. Skill form has two types: an ordered workflow plan, or a declarative ontology of entities, states, and policies. We evaluate the mined skills on two enterprise benchmarks, ThinkingBox-Bench and APEX-Agents. Analysis of task-level paired differences shows that the benefits of different configurations of mining evidence and skill forms depend on the enterprise domain. On ThinkingBox-Bench, paired differences show that workflows score better than ontology by 1.7 pp, Goldilocks beats success-only evidence type by 2.4 pp and Goldilocks blind simulating skills learnt without outcomes is worse by 3.1 pp. APEX-Agents shows a moderate preference for ontologies and no clear preference between evidence regimes. Within each domain, task structure related constraints drive uneven performance with mined skills. These findings motivate tailoring meta-skills to the demands of the target tasks rather than adopting a one-size-fits-all approach.

Mon 5 OctArtificial IntelligenceComputation and Language
The gist
Figuring out how to teach computers to do complicated jobs often involves looking at how past attempts went. The authors studied how different ways of collecting and labeling these past attempts affect the computer's ability to learn helpful skills. They found that which method works best depends on the kind of tasks being done. For some businesses, step-by-step instructions worked best, while for others, more flexible knowledge structures were better. This means there is no one best way to learn skills for computers; instead, you should pick a method that suits the task type.
Open → 2610.05777v1

Agent harnesses pruned to cut token costs without losing power

SHarP: Saliency-based Pruning of Agent Harnesses

Abstract: Agent harnesses are systems that coordinate model calls, tool use, and task execution to help large language models complete complex tasks. To meet task requirements and address failures, these systems are often iteratively refined by amending and patching their instructions, tools, and workflows, continuously increasing harness complexity. It is therefore unclear whether some resulting harness modules are redundant, introducing substantial token overhead with little, if any, performance gain. Inspired by neural network pruning, in this paper, we study harness pruning as a means of striking a better balance between task performance and token cost. We propose SHarP (Saliency-based Harness Pruning), a simple yet effective pruning strategy based on the saliency of each harness module with respect to performance and efficiency. Specifically, we first identify tools, instructions, and supporting mechanisms as components that can be individually disabled. We then estimate the saliency of each module by ablating it and assessing its task performance and token cost relative to the full set of single-module ablations. Modules with the smallest contribution to performance or largest computational overhead are subsequently pruned. Our evaluation across various harnesses on held-out validation sets reveals a surprising finding: most harnesses that we studied are highly redundant and can maintain comparable performance and efficiency even after a substantial portion of their modules are pruned. Our pruning approach and empirical findings provide new perspectives on agent harness design and optimization.

Sat 3 OctArtificial Intelligence
The gist
Agent harnesses are like toolkits that help large language models handle complicated jobs by combining instructions and tools. The group behind this paper found that many parts of these toolkits are unnecessary and only make the system slower and more costly to run. They created a way to measure how important each part is and safely remove the less useful ones. This pruning leads to simpler toolkits that work just as well but use fewer resources. Their findings suggest many agent harnesses can be made leaner without hurting performance.
Open → 2610.04178v1

Llm verification system resolves disagreements in long complex tasks

VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks

Abstract: As LLM agents undertake increasingly complex, long-horizon tasks, verifying their outputs becomes increasingly challenging. We study how verification capability can be strengthened with a fixed base model, without access to reference answers or grading rubrics at test time. Repeated sampling yields multiple rollouts that can contain complementary correct claims, but we need a reliable verification mechanism to determine which claims to trust. We first find that disagreement often exposes correct alternatives, while consensus can conceal errors. These observations motivate VeriHarness, which turns the underlying LLM a generator uses into an agentic verifier by giving it a workspace, evidence tools, and reusable verification skills. A disagreement resolver checks competing claims against environmental evidence, while a consensus challenger tests shared claims and searches for omitted requirements. Their findings guide the selection and revision of the final artifact. Across five long-horizon workspace benchmarks and two frontier models, VeriHarness achieves the highest selection scores among the evaluated baselines. Evidence-backed revision further improves average performance, bringing gains over a single rollout to 6.2 points with Gemini 3.5 Flash and 6.4 points with Claude Opus 4.8. We further show that verification skills can self-improve from failure feedback, demonstrating VeriHarness as a novel and critical approach for scaling long-horizon agentic verification. We release the full pool of approximately 26,000 rollouts across all five benchmarks and both models, produced at a cost of over $100,000, to support future research on agentic verification.

Thu 1 OctArtificial IntelligenceMultiagent Systems
The gist
As language models try to solve big, complicated tasks, checking that their answers are correct is very hard. The paper shows that when multiple model runs disagree, the different answers can help find the truth, while when they agree, mistakes may hide. So the authors created VeriHarness, which helps the model check its own work by examining evidence and testing claims. This method improves how well the final answers are chosen and fixed in five different challenges. VeriHarness can even learn to verify better over time by learning from what went wrong.
Open → 2610.00972v1

LLM collaboration improves with smart model team selection

You're Hired: Strategic Model Selection for LLM Collaboration

Abstract: While multi-agent and model collaboration algorithms gain traction to combine the strengths of diverse Large Language Models (LLMs), existing systems remain bottlenecked on pre-defined and hand-crafted model pools. In this work, we investigate the problem of model selection in multi-LLM systems. We propose and systematically evaluate a taxonomy of 9 selection algorithms ranging from diversity of model descriptions, capability-aware behavioral diversity, and LLM-based recruiters. We conduct extensive experiments across two candidate pools of 10 and 32 models, deployed in four model collaboration algorithms, and evaluated across tasks spanning math, coding, QA, and reasoning. Results demonstrate that successful selection algorithms greatly outperform random or heuristics-based teams such as merely selecting the models with top individual performance, by up to 36.1% across settings. Specifically, capability- and training-based selection strategies alleviate selection variance and achieve the best performance, which we recommend to employ before deploying real-world multi-LLM systems. Further analysis reveals that larger candidate pools pose greater challenges to shallow selection heuristics, while algorithms grounded in interacting with candidate models and understanding model capability robustly filter out misaligned, unsafe models, as well as generalizing to novel, out-of-distribution tasks. Together, we establish that principled and informed team selection is critical and present strong model selection algorithms for assembling effective multi-LLM systems.

Wed 30 SeptComputation and Language
The gist
Combining several large language models (LLMs) can make AI systems better, but choosing which models to include is tricky. The authors studied different ways to select models for these teams and tested nine strategies. They found that picking models based on their abilities and training leads to much better results than just picking the top performers or random choices. This smarter selection helps the system work well even with many candidate models and new tasks.
Open → 2609.38816v1

Self evolving agents learn precise control updates inside workflows

The Right Lesson at the Right Step: Deriving Control Updates for Self-Evolving Agents

Abstract: Self-evolving agents improve future behavior by reusing past experience, typically as global prompts, memories, or reflections. Yet these mechanisms rarely control where experience takes effect. In long tool-use workflows, the same lesson may correct one decision but distract another, making experience reuse a problem of localized control rather than memory alone. We introduce EvoCUE (Evolution through Control Updates from Evidence), a framework for learning reusable control-program updates from completed agent executions. EvoCUE represents the agent as an explicit state-machine controller, whose nodes perform model or tool calls and whose edges define where control passes next. This makes the workflow editable at precise locations, so each learned update can specify what to add, where it acts, and when it applies. From completed trajectories, EvoCUE uses residual goals and observed execution traces to propose localized instruction or skill edits. Each candidate is evaluated at the point where it would act by resuming the parent and edited controllers from the same checkpoint and comparing their final outcomes. Accepted edits are compiled with applicability rules, confirmed on held-out tasks, and inherited by later executions. We evaluate EvoCUE on long tool-use environments where learned conventions must reach the right execution step. From a minimal AppWorld controller without benchmark-specific onboarding instructions, EvoCUE learns the missing task-completion convention and substantially improves success on Test-Normal and Test-Challenge. On PAST-Bench office workflows, EvoCUE transfers organizational requirements from prior episodes to later tasks, improving task-execution quality. These results show that self-evolving agents should place experience inside the control flow, rather than only store it as text.

Mon 28 SeptComputation and Language
The gist
Self-evolving agents usually learn from past actions, but struggle to apply lessons exactly where they help most in a multi-step task. The authors propose EvoCUE, a system that represents an agent’s workflow as a state machine and learns to make precise control changes at specific steps based on past successes or mistakes. By testing edits right where they apply, EvoCUE improves the agent’s future task completion without causing distractions elsewhere. This approach helps agents better transfer knowledge in complex, multi-tool tasks.
Open → 2609.34988v1

FlowState improves long task memory and lowers costs for AI agents

FlowState: Execution State as Memory for Long-Horizon LLM Agents

Abstract: Long-horizon tasks require LLM agents to continually draw on information from earlier interactions. However, retaining the full history increases context costs, while compressing it risks losing details needed later, and the relevance of historical information often becomes apparent as the task progresses. To address these challenges, we propose FlowState, which treats execution state as memory that can be retained and revisited across requests, unifying current decision-making with the reuse of historical information. FlowState preserves semantically typed state nodes, their relations, and references to raw tool observations, separating persistent retention from on-demand access. Within a single execution loop, Incremental State Update (ISU) maintains the current state based on new inputs and feedback, while Progressive State Access (PSA) progressively reveals historical states and supporting evidence as needed during reasoning. Together, these mechanisms enable agents to reassess prior decisions in light of new information and guide subsequent actions. Compared with a full-context baseline using the same DeepSeek-V4-Flash model, FlowState improves the average success rate on MemoryArena and the average pass rate on $τ^3$-Bench by 4.55 and 13.95 percentage points, respectively, while reducing total token consumption by 43.2% and 40.6%. These results demonstrate the performance and efficiency advantages of FlowState on long-horizon tasks.

Mon 28 SeptArtificial Intelligence
The gist
Long tasks need AI agents to remember earlier parts of their work, but keeping all past information is expensive and squeezing it loses important details. The authors introduce FlowState, which stores and updates a special memory that keeps track of important facts and evidence separately. It lets the AI bring up old information only when needed and update decisions as new information arrives. This method improved success rates and cut down the token use by about 40% compared to remembering everything all at once.
Open → 2609.34565v1

Agent boundary method improves safety and action balance in AI agents

AgentBoundary: Counterfactual Evaluation of Safety in Tool-Using LLM Agents

Abstract: Safety alignment for large language models (LLMs) in conversational settings is largely framed around whether to answer or refuse a request. In agentic settings, however, the same models must decide whether to act as permission-critical evidence emerges during execution. This creates a distinct challenge: apparent risk, action permissibility, and task competence are easily confounded, making agentic over-refusal difficult to distinguish from ordinary task failure. To address this, we introduce AgentBound, the first four-way counterfactual generation-and-evaluation framework for tool-using agent safety. AgentBound transforms the same executable workflow by independently varying apparent risk and action permissibility, enabling controlled comparisons of risky-looking but authorized tasks and routine-looking but unauthorized tasks. These comparisons jointly diagnose over-refusal and unsafe compliance while controlling for task competence. We instantiate AgentBound as a human-validated 4,000-task evaluation suite with trajectory-based and post-state-based judgments. Across 17 model and harness configurations, high safety frequently coexists with poor authorized-task completion: GPT-5.5 blocks 99.5\% of routine-looking unauthorized actions yet completes only 28.7\% of risky-looking authorized tasks. We further train a lightweight runtime calibration module that improves authorized-task completion by 18.2\% on average across 10 evaluated configurations, while improving unsafe-action blocking by 5.4\% on average. These show that effective agentic alignment requires action decisions to track permission-relevant execution evidence, rather than refusal strength alone.

Sun 27 SeptArtificial Intelligence
The gist
Large language models acting as agents sometimes refuse to do tasks even when allowed because they confuse risk with permission. The authors created a new way to test and better understand when these AI agents are being too cautious or too risky. They developed a dataset of 4,000 tasks and a method that changes risk and permission factors independently to see how AI decides. Their experiments show that some advanced models often block safe tasks but also prevent unsafe actions. They also trained an additional tool that helps agents do more allowed tasks while still stopping dangerous ones.
Open → 2609.33658v1

Document parsing improved by fixing specific failure cases not complexity

Collapse, Not Complexity: Failure-Conditioned Decomposition Repair for End-to-End Document Parsing

Abstract: End-to-end document parsers increasingly offer an optional reasoning mode for complex pages. On a 180-page entropy-stratified discovery sample with one frozen 4B checkpoint, complexity is the wrong decision variable. Reasoning lowers mean quality by 2.21 Overall at 1.54x tokens; a preregistered input-only model cannot predict its signed benefit (held-out AUROC 0.47, indistinguishable from chance). The benefit concentrates on pages whose ordinary pass has already collapsed, and they do not look complex: shared collapses have lower layout entropy than healthy ones yet consume 19x the tokens as degenerate repetition that doubling the budget does not cure. Switching modes rarely repairs them: 83% recur under reasoning. We instead detect collapse from the ordinary-pass trace, decompose the page by projection, and re-parse each region. Repair gains 1.40 Overall (95% CI [0.68, 2.16]) at 1.13x tokens, replicates across three checkpoints, and, with all parameters frozen, gains 2.41 (CI [1.64, 3.46]) on the remaining 1,175 benchmark pages.

Sun 20 SeptComputer Vision and Pattern RecognitionComputation and Language
The gist
Parsing complex documents can sometimes fail badly, but these failures aren’t always due to the documents being complicated. The authors found that the usual strategy of using more reasoning or increasing effort often doesn’t help. Instead, they show that detecting when parsing fails and breaking the document into smaller parts can fix many errors more efficiently. This new approach improves accuracy while using fewer computing resources compared to traditional methods.
Open → 2609.23592v1

Affora enables software interfaces friendlier for automated agents

Affora: A Design System for Agent-Friendly Interfaces

Abstract: Computer-use agents increasingly operate software designed for people, but interfaces often leave actions or task state unclear to machine readers. We present Affora, a design system that supports both readers while preserving visual freedom and familiar human workflows. Three controlled studies examine component implementations, visual variation, and interaction-design principles. Their findings inform guidance from individual components to complete sites, supported by reusable implementations and executable checks. Agent performance depends on the interaction meaning available through its interface representation; substantial visual variation remains possible when that meaning is preserved. Evaluation on independently authored interfaces shows gains where Affora addresses existing deficits, but limited effects where those deficits are absent or outside its coverage. A workflow case provides preliminary evidence of reduced interaction cost. Affora connects user experience and agent experience through a shared interface rather than a separate agent-only surface.

Wed 16 SeptHuman-Computer InteractionArtificial IntelligenceSoftware Engineering
The gist
Using software usually assumes a human is in control, but now automated agents also interact with interfaces designed for people. The authors introduce Affora, a design system that helps make software interfaces clearer to these agents while still looking familiar to humans. They ran studies showing how different designs affect both humans and agents, and created reusable components that keep the meaning clear even if the look changes. Affora connects what users experience and what agents read through the same interface, improving agent interactions in many cases.
Open → 2609.19125v1

EchoPath enables efficient replay of GUI tasks with image-based targeting

EchoPath: Execution-Level Replayable Memory for GUI Agents

Abstract: Computer-use agents increasingly operate browsers, software, and desktop applications via CLI or API portals, but graphical user interface (GUI) still plays an important role in common industrial production scenarios. GUI agents commonly employ fresh observe-plan-ground-act loops, which is inefficient for enterprise tasks that repeatedly update records, process forms, configure tools, and export reports. We introduce EchoPath, a model-agnostic harness that converts artifact-validated GUI trajectories into standardized, parameter-controlled callable memories, analogous to Model Context Protocol (MCP)-style tool calls rather than unstructured experience records. Each memory stores task-intent keys, application and state preconditions, flexible input parameters, GUI evidence, validation provenance, and lifecycle state, so the host agent invokes a targeted procedure only when it can be deterministically replayed in the current runtime. The core mechanism enabling replay is an image-based target-reaiming algorithm that treats stored coordinates as visual evidence, matches the remembered GUI target against the current screen, and emits corrected operation coordinates before execution. During replay, EchoPath rebinds only declared modifiable inputs and rejects ambiguous or incompatible steps to bounded grounding repair or fresh planning. In experiments with real computer-use tasks, EchoPath reduced median token cost by more than 90% and median execution time by about 60%. These results support a bounded form of enterprise GUI memory: validated execution experience can become a controllable callable asset for recurrent work rather than only context for another reasoning pass.

Tue 15 SeptArtificial Intelligence
The gist
Using graphical user interfaces to repeat tasks on computers is often slow because the software has to figure out what to do every time from scratch. The authors created EchoPath, a system that remembers how tasks were done before by saving images and details about the process. When needed again, EchoPath matches parts of the screen visually to replay the exact steps, making the tasks much faster and cheaper to run. It also only runs steps that it can do reliably, avoiding errors.
Open → 2609.16635v1

Llm agents study new environments without task guidance first

Studying Without a Syllabus: Task-Agnostic Environment Preprocessing

Abstract: Before an LLM agent tackles tasks in a new environment, it can inspect available corpora and tools and construct reusable resources such as indices, scripts, or procedural guidance. Most automated adaptation methods, however, rely on task examples, trajectories, or evaluation feedback to decide what to build. Existing task-agnostic approaches avoid this supervision but commit in advance to a preparation strategy for a particular type of environment. We study a more open-ended setting: can an agent study an unfamiliar environment without a syllabus, i.e. before test time and without knowledge of the downstream task distribution, and choose how to prepare it? We formalize task-agnostic environment preprocessing, in which a studying system explores an environment under a budget and produces artifacts for a frozen solver. We compare unaided and archive-equipped meta-agents with fixed synthetic-practice and corpus-processing methods across six heterogeneous benchmarks. A meta-agent variant achieves the highest Avg@3 reward on five benchmarks, while fixed corpus processing remains best on the largest corpus benchmark. Larger study budgets do not reliably improve downstream reward. Nevertheless, studied artifacts reduce the test-time sampling needed to reach a given score, demonstrating how reusable preparation can shift computation from repeated test-time attempts to a pre-task study phase.

Wed 9 SeptArtificial IntelligenceComputation and LanguageMachine Learning
The gist
When large language model (LLM) agents work in new environments, they usually prepare by looking at examples or feedback tied to specific tasks. This paper explores whether these agents can instead study their surroundings without any task hints, before doing the actual work. The authors created a framework where the agent explores and builds helpful tools without knowing what tasks it will face later. They found that some smart exploration methods improved performance in several test scenarios, although just spending more time studying didn't always help. Their work shows that careful early preparation can save effort later when the agent actually solves tasks.
Open → 2609.10824v1