Hybrid agents struggle to combine web search and databases effectively
Benchmarking Hybrid Deep Research Across Database Querying and Web Search
Computation and Language
Summary
Solving complex problems often means using both the open web and structured databases, but current tests usually look at these separately. The authors created a new challenge called HybridDeepResearch that mixes both web search and database queries. They found that even the best AI models meet this challenge only about half the time, especially when they need to handle one step leading into the next. This shows that getting AI to accurately combine these different kinds of information remains very hard.
What this means in practice
- •For data engineering teams: Evaluate and improve AI systems that interleave database queries with web searches for complex data analysis tasks.
- •For information retrieval developers: Test hybrid agents to better handle transferring constraints between web and database querying in search-based applications.
Authors
Ruofan Wu, Peiran Xu, Xiaolong Li, Fan Shu, Soyoung Yoon, Yite Wang, Xiaodong Yu, Boyi Liu, Feng Yan, Debiao Li, Yuxiong He, Zhewei Yao
Abstract
While autonomous agents have made significant strides in "deep research" by iteratively navigating the open web to synthesize information, real-world problem-solving is rarely confined to a single environment. Complex analytical tasks inherently require agents to weave together evidence from both ambiguous unstructured text (e.g., the open web) and highly precise structured data (e.g., relational databases). However, existing benchmarks evaluate these modalities in isolation, failing to capture the critical "handoff" - the ability to preserve constraints when moving evidence between systems. We introduce HybridDeepResearch, to our knowledge the first deep-research benchmark that requires both web search and SQL to form a complete, verifiable answer. The benchmark contains 380 tool-dependent tasks grounded in LiveSQLBench-Base-Lite databases and public web corpora, validated through automated checks and human review, and covering three reasoning patterns: SQL2S, S2SQL, and Parallel. Evaluations across proprietary and open-weight models under various agentic scaffolds reveal that even state-of-the-art models like GLM-5.2, Claude-Sonnet-4.6 and GPT-5 achieve only about 50-54% Pass@8 on the hard subset. Notably, results show that directional reasoning is substantially more difficult than parallel intersection, highlighting that bridging structured and unstructured information spaces without losing constraints remains a major open challenge for agentic systems. Code and datasets are publicly available at GitHub (https://github.com/Snowflake-AI-Research/HybridDeepResearch) and Hugging Face (https://huggingface.co/datasets/Snowflake/HybridDeepResearch).