LLM driven heuristic design improves by cutting human input
SimpleEvol: An Agent-Loop Framework for LLM-Driven Automated Heuristic Design with Minimal Human Priors
Artificial Intelligence
Summary
Designing heuristics—rules of thumb for solving complex problems—usually involves lots of human guidance and fixed methods. The authors found that relying less on human-designed rules and letting large language models (LLMs) work more independently leads to better heuristics. They created a new system called SimpleEvol that minimizes human interference and lets the LLM take control, resulting in more efficient and improved problem-solving solutions. Their findings suggest that simpler, model-focused approaches might outperform complicated, human-heavy setups.
What this means in practice
- •For software engineers: Automatically generate and improve heuristics for complex optimization tasks with less manual algorithm tuning.
- •For operations research teams: Develop better heuristics for scheduling and routing problems by leveraging autonomous large language model frameworks.
Authors
Jianghan Zhu, Cong Zhang, Rongjie Zhu, Chi Zhang, Zhiguang Cao
Abstract
Large language models (LLMs) have emerged as powerful tools for automated heuristic design (AHD), enabling iterative generation and refinement of heuristics. However, the dominant paradigm embeds LLMs as narrow, fixed components, such as crossover or mutation, within heavily hand-engineered evolutionary frameworks. We argue this misapprehends LLMs. It treats them as specialized tools rather than general reasoners, constrains them to low-level operations, and underutilizes their autonomy. Moreover, the extensive human priors in these frameworks violate the bitter lesson principle that general methods scaling with computation surpass hand-crafted solutions. This raises a key question: which AHD framework designs best convert stronger LLM capabilities into better heuristics? To address this, we propose metrics for LLM-driven AHD framework handcraftedness (AHI) and intelligence conversion efficiency (ICE). Evaluating ten LLMs across three challenging combinatorial optimization problems, we obtain a notable finding that frameworks with fewer human priors consistently yield higher ICE. Based on this finding, we propose SimpleEvol, an agent-loop framework for AHD which removes nearly all human priors and allows the LLM to operate autonomously. SimpleEvol consistently achieves the highest ICE, often by a large margin. Our results challenge the trend toward complex AHD pipelines and point to a lighter and more model-centric alternative, suggesting that reducing human priors is a more effective strategy to scale up with model intelligence. The source code is available at https://github.com/HenryZhu1029/SimpleEvol-Master.