Trust-region radius rules influence optimization step length and cost
A numerical study of radius update rules in trust-region algorithms
Mathematical Software
Summary
Trust-region algorithms are used to find the best solution to a problem by taking steps within a certain size, called the radius. The authors studied different ways to update this step size, showing that the choice of update rule affects the length of each step and the overall cost of finding a solution. They found that while the model used to decide the step direction matters most, the strategy for changing the radius also has noticeable effects. Their work includes tests on designed problems and a package that lets people compare these rules fairly.
What this means in practice
- •For numerical optimization engineers: Design optimization methods with improved step-length control based on robust radius update rules validated on multiple test problems.
- •For mathematical software developers: Implement and test trust-region methods using the provided Julia package to select effective radius update strategies for different problem models.
Authors
Jérémy Rieussec, Fabian Bastin
Abstract
Part I [arXiv:2606.30202] and Part II establish which trust-region radius update mechanisms can fail and under which hypotheses they must succeed. Neither says where the thresholds sit once a rule runs with real constants, or how the rule interacts with the model Hessian and the subproblem solver. This paper turns them into nine falsifiable claims and settles what it can, on four designed problems where the theory's quantities are computable and on CUTEst, where only aggregate cost is visible. Every mechanism runs in one Julia package written for this study, so two runs differing in one component differ in nothing else. The rule is worth less than the model and more than nothing. Crossing the mechanisms with four model Hessians puts $96.5$ per cent of the variation on the model and $1.6$ on the rule, and the rules separate most under the most accurate model rather than least. Uncapped R-grad is the robust default among the criticality-anchored rules, though the retrospective rule is ahead of it on this collection. No constant escapes the side conditions. For every cap there is a minimiser at which the constraint stays active, and for every anchoring constant a flatness on which escape takes arbitrarily long. The second of these is proved here as well as measured, and the proof uses no property of the model or of the subproblem solution beyond $\|s_k\| \leq Δ_k$. The radius rule governs the length of the step, and so the cost of a run. The model decides the search direction, and with it whether the run converges at all.