Classical baselines strongly affect claimed machine-learning speed gains
How the Classical Baseline Shapes Machine-Learning Speed Claims in Topology Optimization
Computational Engineering, Finance, and ScienceMachine Learning
Summary
Topology optimization is about arranging material inside structures to make them strong against forces. Some machine-learning methods claim they find good designs much faster than traditional methods. This paper shows that how well the traditional method is set up and how stiff the designs are matters a lot for these speed comparisons. When the authors carefully matched the strength of designs and improved the traditional solver, the claimed speed improvements often became much smaller or even disappeared. They suggest clearer reporting standards for fair comparison.
What this means in practice
- •For mechanical engineers: Use clearer performance comparisons for selecting topology optimization tools that balance speed and design quality.
- •For software developers of optimization tools: Improve classical code performance and reporting to produce fair benchmarks against machine-learning methods.
Authors
Shaoliang Yang, Jun Wang
Abstract
Topology optimization places material so that a structure resists deformation under given loads. Machine-learning methods often report large speed-ups over classical, non-learned optimization, but a slow classical code or a less stiff learned design can exaggerate them. We re-ran released code for eight methods on one workstation and studied two in depth: Optimize Any Topology (OAT), a diffusion model that generates designs, and a graph-based neural optimizer. Speeding up only the classical code's finite-element solver cut these two methods' speed-ups by up to 20-fold. We then timed the classical code until its design was at least as stiff as the learned one; for OAT, we read this point off the completed runs. Published timings imply a 134x speed-up for OAT on a 256 x 256 grid; our matched single-design comparisons gave 1.5x to 3.5x. At matched stiffness, the graph-based optimizer was slower than the classical code in all six two-dimensional problems. Of the six other methods, two kept their advantage, one shrank, one reversed and two were not comparable. In an exploratory audit of published claims, language models judged that 8 of 28 speed comparisons in randomly sampled, openly accessible articles lacked evidence of equal design quality; our own review counted 11, so the count depends on who judges. We provide a comparison procedure and a minimum report: classical code, hardware, design quality, what each timing includes, and data-generation and training costs.