Proactive verification improves reward reliability for gui agents
VeriCUA: Learning Proactive Verification for Reliable GUI Agent Rewards
Artificial IntelligenceMachine Learning
Summary
Reinforcement learning agents that use graphical user interfaces (GUIs) need rewards that show if they completed tasks correctly. Current methods verify outcomes by asking large language models what to check and when to stop, but they don’t get much better with bigger models. The authors introduce VeriCUA, which learns what to check and when to stop by training a component called the Judge, using only one chance to verify each state. VeriCUA performs better than existing large language models on multiple benchmarks, improving both reward reliability and task success.
What this means in practice
- •For mobile app developers: Improve the accuracy of reward signals for reinforcement learning agents interacting with mobile app interfaces by using a learned verification strategy.
- •For web interface designers: Enhance automated test agents’ reliability when verifying task completion on web platforms through learned, adaptive checking sequences.
Authors
Zhizhou Wang, Yaxi Lu, Zhong Zhang, Zikun Hu, Xin Cong, Yankai Lin, Fangming Liu
Abstract
Reinforcement learning for computer-use agents increasingly relies on proactive reward agents that interact with the post-execution GUI environment to verify what recorded screenshots cannot reveal. The reliability of such verification hinges on two coupled decisions: what to check and when to stop. Existing methods leave these decisions to prompted foundation models. However, neither scaling up the model nor providing few-shot examples reliably improves verification: prompted Judges ranging from 4B to GPT-5.4 all plateau at 68%-70% macro-average agreement. This is because the value of a check depends on the environment rather than the task alone, and becomes apparent only through execution. Learning from execution is itself constrained, as the Actor's terminal state cannot be faithfully replicated during training, allowing only one verification attempt per state. We introduce VeriCUA, a proactive reward agent that learns its verification strategy from these single attempts. A trainable Judge decides what to check and when to stop, while a frozen Probe executes the checks in the live environment. Online reinforcement learning optimizes the Judge for both verdict correctness and efficient Probe use, with returns normalized across terminal states. Trained only on OSWorld and evaluated without further training on MobileWorld and WebArena-Verified, VeriCUA achieves the highest reward agreement across all three benchmarks and lies on the agreement-cost Pareto frontier among proactive verifiers. It surpasses few-shot GPT-5.4 by 5.36 points with 47.6%-59.8% fewer Probe tokens, and even a 1B VeriCUA exceeds every prompted Judge. As the reward for Actor RL, it raises task success by 3.21 points overall and 5.40 points on held-out multi-application tasks. These results identify the verification strategy, not verifier scale, as the key to reliable GUI rewards.