Language model agents verify actions by trying to refute them first
Action Verification through Conjecture and Refutation for Language Model Agents
Artificial IntelligenceComputation and LanguageMachine Learning
Summary
Sometimes language model agents make mistakes because their instructions don't cover everything about a task or environment. Instead of only checking if an action seems right, the authors propose a way for agents to actively look for evidence that an action might be wrong. Their method, BlackSwan, predicts what an action should do, guesses ways it might fail, and then tests for those failures before acting. This approach improved success in tasks where agents interact with simulated worlds and websites, showing better self-correction abilities.
What this means in practice
- •For software developers: Improve task automation reliability by enabling agents to test and reject faulty actions before execution in interactive applications.
- •For ecommerce platform operators: Enhance user-assisting agents on shopping websites by reducing incorrect actions through active refutation-based verification.
Authors
Minhyeon Oh, Chiyeong Heo, Geonwoo Noh, Hoyoung Kim, Jangwon Son, Jungseul Ok
Abstract
Language model agents increasingly act on users' behalf by manipulating external environments through tools, often from high-level instructions that leave critical task- and environment-specific context unstated. To prevent resulting misaligned actions, recent work verifies proposed actions before execution, largely by establishing whether the conditions supporting them hold. We argue that this confirmation-oriented view leaves a fundamental blind spot: a wrong action often passes every check that asks whether it is right. This motivates a different question for verification: rather than asking what evidence supports an action, can the agent find evidence that refutes it? We pursue this direction with BlackSwan, a refutation-oriented, expect-suspect-test procedure that predicts the intended effect of each state-mutating action, formulates specific and empirically testable hypotheses for how it could be wrong, and tests them using existing observations or additional non-mutating actions, executing the action only if none survives testing. On challenging AppWorld tasks, BlackSwan improves success rate over the strongest baseline by 14.9 and 25.2%p with and without the backbone model's thinking mode, respectively. Even without thinking or in-context demonstrations, it outperforms stronger baselines equipped with them. The gains also transfer beyond AppWorld, with BlackSwan improving WebShop performance by 12.7%p. More broadly, our results suggest that language model agents have an underused capacity to recognize and self-correct misaligned actions, which can be systematically elicited through active conjecture and refutation before execution.