Evaluating Real-World Generalizability of Algorithm Selection Models
2026-06-01 • Machine Learning
Machine Learning
AI summaryⓘ
The authors studied how well algorithm selection (AS) models work when used on different types of optimization problems, including both made-up test cases and real-world tasks like robot path planning. They tested AS models on two common benchmark sets and two practical problem sets to see if the models could pick good algorithms across these varied scenarios. Their analysis showed where the models did well and where they struggled, especially when moving from academic tests to real-world problems. This helps understand the limits of current AS methods and points toward building better ones for real-life use.
Algorithm SelectionOptimizationBenchmark SuitesBBOBCECRobotics Trajectory OptimizationUnmanned Aerial Vehicle Path PlanningCross-benchmark EvaluationGeneralizationPerformance Prediction
Authors
Gjorgjina Cenikj, Jakub Kudela, Eva Tuba, Tome Eftimov
Abstract
Algorithm Selection (AS) aims to automatically identify the most suitable optimization algorithm for a given problem instance by leveraging measurable problem characteristics and historical performance data. In this study, we investigate the generalization ability of AS models across both synthetic and real-world optimization landscapes. We consider two widely used academic benchmark suites (BBOB and CEC) and two real-world problem sets (robotics trajectory optimization tasks and unmanned aerial vehicle path-planning problems). Through a systematic cross-benchmark evaluation, we analyze how AS models transfer between domains, identify where generalization succeeds or breaks down, and highlight the challenges that arise when applying AS in realistic, domain-specific contexts. Our findings provide insights into the robustness of current AS approaches and inform the development of more reliable, broadly applicable AS systems for real-world optimization.