Agentic Auto-Research is Fuzz Testing
2026-08-10 • Artificial Intelligence
Artificial IntelligenceComputation and Language
AI summaryⓘ
The authors explain that when computer programs try to do research by creating experiments on their own, they often produce more ideas than humans can check. Current methods mainly generate many experiments and then pick the best ones, but the authors say this misses a key point: the program needs useful clues or feedback from each experiment to guide what to try next. They suggest that experiments should give ongoing, easy-to-get signals about progress before final validation, so the program can search smartly instead of guessing randomly. They also recommend testing whether this feedback helps find more true discoveries efficiently while avoiding false results.
autonomous research agentsgenerate-and-rank paradigmgreybox fuzzingsparse feedbackepistemic progressfeedback-directed searchfinal validationadaptive reusefalse discoveries
Authors
Yifeng He, Jicheng Wang, Yinzhe Zhao, Jiachen Liu, Hao Chen
Abstract
Autonomous research agents can generate experiments faster than researchers can validate them. Researchers have responded by scaling the proposer and ranking more samples with a learned judge or human reviewers. We argue that this *generate-and-rank* paradigm misses the problem of sparse feedback. Within a declared research problem, an agent follows the control loop of a greybox fuzzer: it proposes a candidate, executes it, observes feedback, and chooses what to try next. A fuzzer rarely finds a bug, but coverage makes partial progress observable on every execution. Fuzzers then use that signal to mutate inputs and allocate effort, rather than only to rank completed runs. Auto-research needs the same two capabilities. First, each experiment should expose a cheap, dense signal of epistemic progress before final scientific validation is available. Second, that signal should determine the next intervention so that the agent searches rather than repeatedly samples. Because the optimized progress signal is guidance rather than a verdict, final validation must still decide what counts as a discovery using evidence protected from adaptive reuse. We propose controlled tests of whether candidate signals predict validated progress, whether feedback-directed search yields more validated discoveries per unit cost than repeated sampling, and whether protected validation reduces false discoveries. Feedback architecture, not only generation, is a central bottleneck in auto-research.