Semopt improves fixing errors in automated optimization models
SemOPT: Fixing Semantic Errors in LLM-based Optimization Modeling via Reward-Guided Search
Computation and Language
Summary
Optimization modeling helps turn real-world problems into math problems computers can solve. The authors found that when large language models (llms) automatically create these math models, they often make subtle mistakes that still run but give wrong answers. To fix this, they built semopt, a system that scores how well a generated model matches the original problem’s meaning and searches for better versions. Their tests show semopt does a noticeably better job correcting these tricky mistakes.
What this means in practice
- •For energy planning teams: Improve automatic generation of optimization models for energy resource allocation by detecting and fixing hidden semantic errors.
- •For healthcare operations managers: Enhance accuracy of optimization-based scheduling tools by automatically correcting model misinterpretations that don’t cause runtime errors.
Authors
Zetong Zhou, Wentao Zhang, Jingyuan Wang, Yifan Yang, Zizhuo Wang, Shixi Hu
Abstract
Operations research supports decision-making in domains such as energy, economics, and healthcare. Solving operations research problems typically begins with optimization modeling, which translates a natural-language problem description into executable solver code. LLMs offer a promising way to automate this process, but they remain prone to errors. In practice, these errors can be divided into two categories: syntactic errors refer to solver code that fails to run successfully or is judged infeasible by the solver; semantic errors refer to solver code that successfully returns an objective value but violates the intent of the original problem. Since semantic errors do not trigger runtime failures, they are difficult to detect and rectify. To address this problem, we introduce SemOPT, a semantic-guided framework for correcting LLM-based optimization models. SemOPT combines a semantic reward model that distinguishes faithful math models from plausible but incorrect ones with an adaptive correction system that applies hierarchical reward-guided search over the modeling space. Experiments on seven optimization modeling benchmarks show that SemOPT establishes a new state of the art and achieves an average 7.6% accuracy improvement over the strongest baseline on complex datasets.