Hywe unifies web search and interaction for complex online tasks
HyWe: Unifying Search and Operation for Real-World Hybrid Web Tasks
Artificial Intelligence
Summary
Many real-world web tasks need both finding information and clicking or typing on web pages, but usually these abilities are handled separately. The authors created HyWe, a single system that switches between searching and interacting with pages to complete tasks. They trained this system on examples that combine these skills, and it performs better on new tasks than existing tools that only search or only interact. This approach helps make web automation smarter and more flexible.
What this means in practice
- •For web automation engineers: Build assistants that can both search web content and directly interact with webpages to complete complex tasks more effectively.
- •For digital customer support teams: Create automated agents capable of retrieving information and performing browser actions to assist users across diverse websites.
Authors
Zijian Cai, Haowei Liu, Kailai Chen, Lin Peng, Cong Wan, Ziqi Zhang, Chunfeng Yuan, Bing Li, Weiming Hu, Haiyang Xu
Abstract
Real-world hybrid web tasks need both information gathering and page interaction, yet search-based and operation-based agents are developed and evaluated separately. To bridge this gap, we introduce HyWe, a framework that unifies both capabilities within one model and trajectory. Its two modes use dedicated prompts and tool sets to sustain sub-goal execution; explicit switching selects the needed capability, and shared history lets each mode build on the other's findings. To provide demonstrations of these behaviors, we develop HyWeCollector, a fully automatic pipeline that uses task prototypes, four switching patterns, and multi-agent synthesis to produce verified hybrid trajectories. We train HyWe-35B on these demonstrations to unify efficient information gathering and browser interaction in a single 35B web agent. To evaluate hybrid task completion, we introduce HyWeBench: 100 human-written tasks on websites distinct from training, with human-written sub-goal checklists and an outcome-based evaluation pipeline. Hybrid training enables HyWe-35B to achieve 47.0% success on this benchmark, improving on the untrained base model by 19.0 percentage points and outperforming existing search-based and operation-based agents. Even before training, the same base model solves 28.0% of tasks with HyWe, versus 11.0% with all tools under one prompt. Beyond hybrid tasks, HyWe-35B achieves 59.0% and 88.3% success on Online-Mind2Web and WebVoyager, outperforming existing open-source baselines and cutting action cost per successful task by 48.3% and 51.9% compared with Fara-7B.