Self-evolving search improves reliability with statistically calibrated reward intervals
Reliable Self-Evolution with Imperfect Proxy Rewards
Artificial Intelligence
Summary
Sometimes, computers try to discover new scientific ideas by generating many possibilities and then choosing the best ones. But checking if each choice is truly good can be very expensive, so they use quick and cheap guesses that aren't always correct. The authors created a method called CISE that uses smart statistical tools to better understand how reliable these guesses are and only picks choices that are very likely to be truly good. This helps avoid wasting time on bad ideas and makes the whole search process more trustworthy.
What this means in practice
- •For materials engineers: Generate reliable material candidates with guaranteed property feasibility under limited costly validation resources.
- •For automation engineers: Enhance automated design processes by filtering out infeasible options using interval-based reward confidence.
Authors
Kangjun Noh, Soyu Kim, Kyungwoo Song
Abstract
Large language model (LLM)-based self-evolving search is a promising approach to scientific discovery. However, high-fidelity evaluation of every candidate is prohibitively expensive in some domains. Self-evolving systems in such settings therefore rely on low-cost but imperfect proxy rewards, which may assign high scores to infeasible candidates. These false positives may contaminate both the final output and the feedback used to guide subsequent generations. This motivates statistically calibrated reward intervals for more reliable self-evolving search. We propose Conformal Interval-Driven Self-Evolution (CISE), which constructs candidate-specific reward intervals using conditional conformal inference and iteration-wise online density-ratio estimation. CISE uses conservative interval-based rewards for evolutionary feedback and returns candidates only when all required property intervals lie entirely within their respective feasible regions. We derive fixed-iteration coverage results under explicit assumptions of independence and covariate shift. We evaluate CISE on three self-evolving search tasks in materials science. In our experiments, all candidates returned by CISE are true positives under high-fidelity evaluation, whereas the baselines return more candidates but include false positives. These results highlight the value of a smaller, more precise shortlist when downstream validation budgets are limited. Our repository is available at https://github.com/MLAI-Yonsei/CISE.git.