Evaluating web searches that create 3D worlds from user requests
Search-to-World: Evaluation of 3D World Delivery from User Request through Web Search
Computer Vision and Pattern Recognition
Summary
It can be hard for smart systems to turn what they find on the internet into 3D worlds you can see and explore. This paper introduces a way to test how well these systems do that, by checking if they find useful content and then build a good 3D world from it. The authors also build a system called WorldSearcher that tries to reuse existing 3D worlds or make new ones from videos, with tools to fix problems along the way. They found that just finding relevant information doesn't guarantee a successful 3D world, but training recovery tools helps make better worlds more efficiently.
agentic systems3D world deliveryweb searchvisual content retrievalObserved Retrieval RateWorld Delivery Ratefine-tuningreconstructiontemporal groundingquery reformulation
Authors
Zixiao Gu, Yabo Chen, Xunzhi Xiang, Yu He, Haibin Huang, Chi Zhang, Yunbo Wang, Xuelong Li
Abstract
Agentic systems can interpret user requests, search the live web, and use external tools, but their ability to transform retrieved web content into a usable 3D world has not been systematically evaluated. No established end-to-end pipeline or benchmark exists for this capability. We introduce Search-to-World, an end-to-end evaluation task covering request understanding, web visual-content retrieval, and 3D-world delivery. We define Observed Retrieval Rate (ORR) and World Delivery Rate (WDR) to distinguish observing relevant content from successfully delivering a request-aligned, perceptually acceptable world. We also present WorldSearcher, a reuse-then-reconstruction harness that connects existing search agents to world delivery: it first retrieves reusable 3D worlds and, when none are available, reconstructs a world from video. A structured recovery controller revises temporal grounding, replaces source videos, or reformulates queries after failure. Using WorldSearcher, we benchmark representative models on Search-to-World and study supervised fine-tuning (SFT) for recovery subagents. Results show that delivery depends on the underlying agentic model, and that relevant-content observation does not ensure world delivery. Jointly training recovery agents improves delivery success and action efficiency. Search-to-World makes agentic 3D-world delivery measurable, while WorldSearcher provides a practical evaluation harness with recovery capabilities.