Lapanda speeds up solving complex optimization problems with constraints
lapanda: A Matrix-Free Differentiable Solver for Nonconvex Constrained Optimization Layers
Machine Learning
Summary
Solving complex math problems that have rules to follow can be slow and use a lot of computer memory. The authors created lapanda, a tool that solves these problems faster and with less memory by avoiding heavy matrix calculations. It uses a step-by-step method that breaks the problem into smaller parts and efficiently finds solutions while keeping track of rule satisfaction. Tests show that lapanda works well in tasks like robot movement and controlling systems, using less time and memory than other tools.
What this means in practice
- •For robotics engineers: Optimize robot motion planning under constraints like obstacle avoidance more efficiently in embedded systems.
- •For control system developers: Improve training of constrained optimal control models by faster and memory-efficient differentiable solvers.
Authors
Yuankun Chen, Zifei Nie, Kangyu Lin, Ján Drgoňa, Liang Wu
Abstract
Differentiable optimization brings the structural guarantees of mathematical optimization to network pipelines, allowing them to be trained end-to-end. However, its application remains challenging for nonconvex constrained problems, as existing differentiable solvers often suffer from limited modeling expressiveness due to their reliance on specialized problem structures, while also incurring substantial computation time and memory overhead in both the forward and backward passes. To address these challenges, we propose lapanda, a matrix-free differentiable solver for nonconvex optimization with general constraints. It reformulates the problem to a sequence of augmented Lagrangian subproblems, each handled by a first-order inner solver through a proximal averaged quasi-Newton algorithm with adaptive linesearch, thus enabling efficient forward optimization. We establish local well-posedness of the solution map and convergence of the outer iterations, and further derive a sensitivity alignment between the original problem and the final subproblem in the backward pass, demonstrating that the subproblem sensitivity, which can be computed efficiently in a matrix-free manner, provides a principled approximation to the exact optimizer sensitivity. We evaluate lapanda on nonconvex constrained Rosenbrock benchmarks, imitation learning with several representative constrained optimal control problems, and embedded robotic obstacle-avoidance tasks. Compared with state-of-the-art differentiable solvers, lapanda delivers substantial reductions in computation time and memory footprint while maintaining reliable constraint satisfaction and learning performance.