Tool-using agents learn when to act or ask for help
Agents' Last Doubt: Knowing When to Act and When to Step Back in Tool-Using Agents
Artificial Intelligence
Summary
Tool-using AI agents often face tricky choices about whether to act on their own, gather more information, or ask a human for help. The authors created a special dataset that records outcomes not only for actions taken but also for all possible alternative actions to understand these choices better. They used this data to train a controller called SURE-Act that helps agents decide the best move in uncertain situations. Their tests show this approach improves the agent’s performance and beats other decision-making methods. This work helps AI agents make smarter decisions when handling tools to avoid mistakes or unnecessary delays.
What this means in practice
- •For ai developers: Improve AI assistants’ decisions about when to act or ask for help to balance success and risk.
- •For customer support platforms: Enhance automated support bots by enabling smarter escalation decisions to human agents, reducing errors and unnecessary queries.
Authors
Qisai Liu, Qingyue Jiao, Bangwei Guo, Wenxi Dai, Changyi Yang, Jiahe Liu, Dimitris N. Metaxas, Yiyu Shi, Zhanhong Jiang, Minghao Guo
Abstract
Tool-using LLM agents repeatedly face decision points under uncertainty, choosing between autonomous action, additional information gathering, and human escalation. These decisions require balancing task success against interaction cost and the risk of irreversible errors. Learning to make these trade-offs is challenging because existing interaction trajectories record only the outcomes of actions taken, leaving the consequences of alternative actions unobserved. To address this gap, we construct a counterfactual dataset containing 3,115 action outcomes from 1,036 decision points in ToolSandbox. At each decision point, we execute each legal action and evaluate its downstream outcomes based on task completion, interaction cost, and irreversible errors. Using this dataset, we train SURE-Act, a controller that selects actions for a frozen backbone agent at inference time. Our experiments show that learning to rank alternative actions improves offline action-selection utility by 0.0309 over a regression-only baseline. We further show that adding SURE-Act enables end-to-end gains of up to 0.0575 in ToolSandbox's Overall Score with the Qwen3-4B agent. Moreover, SURE-Act outperforms adapted decision rules from SAGE-Agent and Calibrate-Then-Act by 0.1506 and 0.1438, respectively. This work establishes the first counterfactual dataset for ToolSandbox and provides actionable insights into decision-making under uncertainty for tool-using agents.