Risk-sensitive planning improves multi-target satellite inspections in orbit
Risk-Sensitive Mission Planning for Multi-Target On-Orbit Inspection via Distributional Reinforcement Learning
Artificial Intelligence
Summary
Planning missions to inspect multiple satellites in space involves uncertainty and risks that can affect safety and success. This paper shows how using a special type of machine learning, called distributional reinforcement learning, helps plan these missions by considering both the expected success and the variability of outcomes. By adjusting risk sensitivity, the planning method can reduce the chance of mission failure while still achieving good results. The authors tested their approach in simulated space missions and found it makes safer inspection plans without losing much in performance.
What this means in practice
- •For satellite mission planners: Plan safer multi-target inspection campaigns for satellites by balancing expected mission success and risk of failure in real-time.
- •For spacecraft control engineers: Design adaptive onboard decision systems for low-thrust servicing spacecraft to select inspection targets considering changing risks and mission time limits.
Tested on simulated data.
Authors
Adam Abdin
Abstract
Multi-target on-orbit servicing missions require planning tools that can adapt to evolving operational conditions while accounting for the outcome variability inherent to safety-critical spaceflight. This paper presents an adaptive planning approach for multi-target servicing campaigns in low Earth orbit and applies it to inspection missions flown by a low-thrust servicer. The approach is based on Distributional Reinforcement Learning, which learns the full distribution of mission returns and thereby supports explicit risk-sensitive decision-making at each planning step. Specifically, a Quantile-Regression Deep Q-Network (QR-DQN) agent selects which target to inspect next by jointly considering the expectation and the variance of the predicted return through a mean-variance criterion. The mission environment models circular orbits with a spread in inclination and uses Edelbaum's combined-manoeuvre low-thrust transfer estimates. The observation comprises the orbital elements, the visit history, the observed risk state of each target and the remaining mission-time budget. The risk state of each target changes stochastically during the campaign, making the outcome of an inspection uncertain and creating operationally meaningful distinctions between risk-neutral and risk-averse planning strategies. Because the mission-time budget generally admits only a subset of the candidate targets, the planner must also select which targets to inspect. Numerical experiments on scenarios with five target satellites show that one risk-aversion weight moves the policy across a range of mean return and return variance, over an unchanged environment and reward. Risk-averse agents reduce the variance of the episodic return by approximately one half relative to risk-neutral agents and lower the probability of a mission loss, at a cost in mean return. To validate the learned policies, the efficient frontier, which gives the highest mean return attainable at each level of return variance, is computed exactly by enumerating the admissible visit orders. At equal variance, the learned policies lie close to this frontier in mean return, and the risk-averse policies have a lower probability of a mission loss than the frontier policies.