Reinforcement learning optimizes satellite constellation deployment under uncertainty
Offline Approximate Value Iteration: A Reinforcement Learning Approach for Satellite Constellation Deployment
Artificial Intelligence
Summary
Deploying groups of satellites involves complicated decisions, especially when problems like launch failures or broken satellites can happen. The authors treat this task like a decision-making game that takes these uncertainties into account. They use a method called approximate value iteration, which simplifies the problem to find good deployment plans without requiring too much computer power. Their early tests show these plans could be faster and cheaper than existing strategies for getting satellites into orbit.
What this means in practice
- •For satellite mission planners: Generate efficient deployment schedules for satellite constellations that account for launch and satellite malfunctions.
- •For space logistics teams: Reduce computation time in planning satellite deployments under uncertain conditions using simplified reinforcement learning methods.
Authors
Tristan Leuridan, Yiping Fang, Adam Abdin
Abstract
This paper introduces a reinforcement learning-based approximate dynamic programming approach to optimize the deployment of Walker-type satellite constellations under uncertainties, including launch failures and satellite malfunctions. We formulate the problem as a Markov Decision Process that incorporates low-injection orbit launches, continuous low-thrust orbit raising, and variable satellite loads. We adopt the offline approximate value iteration solution method, using linear function approximation with basis functions and action pruning strategies to substantially reduce the computational and memory requirements for finding optimal decision policies. Preliminary results demonstrate convergence to policies with low expected costs and deployment times, both of which are lower than those of a rolling-horizon baseline, showcasing a novel application of approximate dynamic programming in space logistics.