Planning safely inside learned models with convex optimization

Successive Convexification in Learned World Models for Safe Offline Planning

Machine LearningRobotics

Summary

Planning actions safely using data from past experiences is hard because approaches often guess many possible plans and reject unsafe ones, which can be inefficient. The authors propose a new method that turns this into a step-by-step mathematical problem with safety rules built in. They smooth the model’s behavior to make this step-by-step approach work well and start it from a guess to find safer plans faster. In tests with simulated navigation tasks, their method found safer paths with fewer rule violations and gave a clearer answer if no safe plan was nearby.

What this means in practice

  • For robot navigation teams: Plan safe robot movements using learned models without extra data collection, reducing unsafe actions during navigation in complex environments.
  • For automated vehicle developers: Improve offline planning for safety-critical decision making by integrating convex optimization that verifies plan feasibility stepwise within learned models.

Tested on simulated data.

Authors

Samuel Younger

Abstract

World models learned from offline data allow an agent to plan without further interaction with the environment, but the sampling-based planners used with them handle safety constraints only by scoring or rejecting sampled action sequences. We instead pose safe planning in the latent space of a learned world model as a successive convex program (SCP), in which a learned discrete-time control barrier function is imposed as a constraint at every step of the horizon. SCP requires smooth dynamics, so we build the networks from smoothed MaxMin activations and spectrally normalised layers, and bound the resulting linearisation error in terms of the Lipschitz bound and the smoothing width. As SCP is a local method, we start it from a sampled plan. On two Safety-Gymnasium navigation tasks with offline data, this planner has fewer violations than MPPI and CEM drawing 128 times as many samples at equal wall-clock time in our main comparison, though not significantly, and earns greater return at longer horizons, and is much safer than SCP without a sampled start. SCP converged more often on Lipschitz-bounded models. Unlike sampling, it also reports at each step whether its plan is a feasible, first-order optimal KKT point of the planning problem in the model, or that no feasible plan was found nearby. Our implementation is publicly available at https://github.com/youngers2006/scwm.