Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation
2026-07-06 • Computer Vision and Pattern Recognition
Computer Vision and Pattern RecognitionArtificial Intelligence
AI summaryⓘ
The authors show that current image generators often make mistakes when asked to create visuals about new or rare topics because they are trained on fixed data and can't keep up with the changing world. They created a large dataset to test these failures and found that simply using search engines with these generators doesn't help much because irrelevant information confuses the system. To fix this, the authors propose a method where the generator learns what it can already know and what it needs to search for, improving its results step-by-step. They provide all their data and tools so others can build on their approach for better world-aware image generation.
visual generatorsworld knowledgelong-tail distributionsearch augmentationco-trainingbenchmark datasetmultimodal dataknowledge boundaryagentic generationtool-augmented generation
Authors
Haozhe Wang, Weijia Feng, Jinpeng Yu, Che Liu, Ping Nie, Fangzhen Lin, Jiaming Liu, Ruihua Huang, Jimmy Lin, Wenhu Chen, Cong Wei
Abstract
Visual generators excel at rendering, but they confidently fabricate what they do not know. User requests are unbounded, evolving, and deeply long-tailed: new characters, trending entities, post-cutoff events, and more. This world-knowledge bottleneck is structural: generators are trained on fixed corpora, but the visual world is open-ended. We construct SearchGen-20K and SearchGen-Bench, with 20,839 prompts spanning twelve failure categories and twenty-two domains, paired with a pre-executed multimodal SearchGen-Corpus-1M to support offline, reproducible research. On SearchGen-Bench, frontier open generators score only 21 to 28 out of 100, a 40-point collapse invisible to existing benchmarks. The natural remedy is to employ search tools, enabling agentic visual generation. However, we find that naive search fails: it retrieves indiscriminately, injecting noise into prompts the generator already handles. We trace the root cause to a generator-specific, evolving knowledge boundary: the divide between what a generator can internalize through training and what must remain in external context. Although this boundary is hard to specify in advance, we show that it is discoverable through a teach-then-search co-training framework. Even a minimal version of this co-training recipe produces monotonic improvement, laying the foundation for recursive self-improvement in visual generation that can meet world-knowledge-grounded requests. We release the full dataset, co-training corpus, and search corpus as a replayable harness for tool-augmented, world-knowledge-grounded visual generation.