Web agents struggle to bypass access barriers and preserve goals
AgentHeal: Evaluating Goal-Preserving Resource Recovery of Web Agents under Access Barriers
Cryptography and SecurityComputation and Language
Summary
Web agents that use large language models are good at handling tasks online but often get stuck when faced with things like dead links or login pages. The authors built AgentHeal, a test to see how well these agents can find alternate ways to still complete the user's task without breaking rules. Their tests showed that success rates drop a lot when these barriers appear, as agents sometimes retry uselessly, fall for fake sites, or ignore the user's limits. This research helps pinpoint how to make smarter web agents that keep working even with tough access limits.
What this means in practice
- •For web automation developers: Build web agents that better detect and reroute around blocked resources to fulfill user goals despite access restrictions.
- •For enterprise software teams: Improve reliability of automated information-gathering tools by diagnosing failure modes under access barriers like authentication and rate limits.
Authors
Zhiyi Liu, Haohua Du, Haoqi Sun, Haisheng Tan
Abstract
Large language model (LLM) agents have demonstrated strong capabilities in automating complex information-seeking and transactional tasks across the open web. However, their real-world reliability remains limited by access barriers, such as dead links, authentication walls, rate limits, and anti-scraping defenses. These barriers block the primary execution path, causing agents to abort prematurely, loop indefinitely, or quietly compromise the user's original constraints. Existing evaluations either implicitly assume that target resources are freely accessible or focus on transient perturbations that simple retries can resolve, failing to capture scenarios where agents must actively reroute around impassable bottlenecks. To bridge this gap, this paper introduces AgentHeal, a comprehensive benchmark evaluating goal-preserving resource recovery, defined as the ability of web agents to detect non-transient access barriers, pivot to legitimate alternative sources, and fulfill the user's exact original objective. Spanning nine barrier families across a counterfactual sandbox and real-website replay, our experiments on leading LLM backbones show that average task success drops sharply from 89.4% to 43.7% under access barriers. Further analysis reveals that agents frequently fall for deceptive look-alikes, fail to escalate to users when necessary, violate access policies, or waste computation on futile retries, providing actionable diagnostics for developing resilient web agents.