Foundation models enable adaptive space robot perception and action
Foundation Models for Autonomous Space Robotics: Architectures, Physical Adaptation, and Validation
Robotics
Summary
Space robots face many challenges like unusual conditions and limited human help. The authors review how large foundational AI models can help robots perceive, plan, and act in space. They discuss specific tools like event cameras and special neural networks that could improve sensing and control. The paper also highlights how to test these models on the ground before trying them in space. Overall, it shows what foundation models can already do and what testing is still needed for space use.
What this means in practice
- •For spacecraft control teams: Use foundation models combined with event cameras to improve autonomous satellite capture and proximity operations under uncertain conditions.
- •For robotics hardware designers: Develop energy-efficient onboard processors that utilize spiking neural networks for sparse, local computation in space robots.
A survey. It maps existing work.
Authors
Zizhe Wang, Jia Deng, Xin Lu, Zehao Jia, Yi Huang, Yao Zhu, Jianing Li
Abstract
Space robots must interpret tasks and act despite unfamiliar dynamics, difficult sensing conditions, and limited opportunities for human intervention. Foundation models offer reusable perceptual and behavioral knowledge, but their value depends on how that knowledge reaches physical execution. This critical review follows space-specific requirements through perception, prediction, planning, and action to deployment and validation, emphasizing orbital manipulation and proximity operations. Terrestrial comparisons establish benefits of particular pretraining stages and model interfaces; their transfer to space dynamics and constrained onboard operation remains insufficiently tested. Event cameras offer timely motion observations across strong illumination contrasts, while spiking neural networks provide possible routes to sparse local computation. We examine how sensor degradation, state estimation and computing choices affect feedback and recovery. Higher action throughput need not yield fresher feedback; lower processor power can also coexist with higher energy per inference. We organize validation as an iterative sim-to-ground-to-space pathway, connecting physical tests with telemetry-based diagnosis and closed-loop revalidation. A satellite-capture example links observations to approach, contact and recovery, while a corresponding protocol separates generalization from adaptation with limited target-domain interaction. These comparisons identify what models already contribute and the physical evidence needed to support autonomous space operation.