Reinforcement learning improves spacecraft interplanetary maneuver planning
Reachability-Informed Reinforcement Learning for Multi-Impulse Interplanetary Transfers
Machine Learning
Summary
Planning spacecraft trips between planets involves choosing a sequence of maneuvers that use as little fuel as possible. The authors developed a way to teach computers to make these decisions by combining knowledge about reachable points in space with reinforcement learning. Their method helps decide intermediate stops during a trip, ensuring maneuvers are physically accurate and efficient. Tests show their approach produces near-optimal plans that work well even when starting conditions vary.
What this means in practice
- •For space mission planners: Generate efficient sequences of spacecraft maneuvers for interplanetary trips with adaptable plans that accommodate varied departure conditions.
- •For aerospace software developers: Incorporate reinforcement learning driven waypoint selection to improve trajectory planning tools combining classical astrodynamics with learned decision models.
Authors
Yashdeep Chaudhary, Roberto Armellin, Harry Holt
Abstract
Reinforcement learning offers the prospect of a reusable sequential decision-making mechanism for spacecraft trajectory design, motivating policy interfaces that connect learned decisions to the underlying maneuver geometry. This paper develops Reachability Analysis-Informed Reinforcement Learning (RARL) for deterministic multi-impulse interplanetary transfers, placing intermediate waypoint selection at the center of the learned decision process. Local first-order reachability maps bounded velocity perturbations into an ellipsoidal set of next-node positions, within which the policy selects its waypoint. Lambert reconstruction then determines the corresponding maneuver to reach this selected waypoint along a dynamically consistent ballistic arc, coupling learned transfer-geometry selection with classical astrodynamics. A terminal two-impulse reconstruction completes the rendezvous, supported by a linear maneuver-demand assessment used for reward shaping. Numerical studies characterize this interface on a two-body Earth-Mars benchmark. Across three independent training runs, RARL achieves a mean maneuver cost of 10.23 km/s, 1.72% above a validated local sequential convex programming reference. Training over dispersed initial states extends policy reuse across a departure family with fixed target state and transfer duration. Each of the three independently trained multi-state policies completes all 10,000 held-out Monte Carlo departures without impulse-cap violations, compared with a mean feasibility rate of 6.49% for single-state policies. This broader sampled feasibility is accompanied by a 0.61% increase in mean nominal maneuver cost, without further training across departures. These results demonstrate that a reachability-informed decision interface supports benchmark-quality trajectory construction and policy reuse across dispersed departure conditions.