DreamSat-Bench creates platform to test AI spacecraft pose estimation
DreamSat-Bench: Development and Initial Testing of a Testbed for AI-Based Pose Estimation from 3D Reconstruction
Robotics
Summary
It is hard for spacecraft to figure out where other nearby spacecraft are, especially without prior detailed models. The authors built DreamSat-Bench, a test setup that uses both computer simulations and real robotic arms to mimic spacecraft movements and lighting conditions. This platform helps test AI techniques that estimate the position and orientation of other spacecraft from 3D images. Their experiments show how factors like image delay and lighting affect accuracy and identify challenges like tracking drift. This tool aims to improve autonomous space navigation when little is known about other satellites.
What this means in practice
- •For spacecraft navigation engineers: Benchmark AI-based relative navigation methods for autonomous rendezvous and proximity operations with unknown satellites.
- •For robotics system developers: Test and improve robotic vision systems that transition from simulation to real-world environments using custom orbital scenarios and lighting.
Authors
Alex Posadas-Nava, August Berne, Giovanni Lavezzi, Kareena Shah, Alejandro Carrasco, Josiane Uwumukiza, Giacomo Battaglia, Paolo Panicucci, Minduli C. Wijayatunga, Victor Rodriguez-Fernandez, Richard Linares
Abstract
This paper presents the development and initial testing of DreamSat-Bench, a modular rendezvous and proximity operation testbed designed to benchmark AI-based relative navigation techniques. By integrating a software- and hardware-in-the-loop robotic pipeline, the platform enables a seamless transition from digital simulation to physical reality. DreamSat-Bench unifies state-of-the-art robotic learning tools such as MuJoCo, Isaac Lab, and LeRobot into a single benchmarking platform, utilizing robotic arms to trace 3D trajectories. The platform allows for extensive customization of orbital environments and lighting to evaluate the simulation-to-reality gap. We demonstrate the testbed's utility by evaluating an end-to-end vision-based navigation pipeline that pairs DreamSat, a generative AI framework for single-view 3D reconstruction, with FoundationPose for zero-shot 6-DoF tracking of unseen spacecraft. Initial testing explores mission-representative orbital segments, including fixed-point station-keeping and fly-around characterization. Through a series of parametric studies, we quantify the impacts of reconstruction latency, mesh resolution, orbital range, and illumination geometry on pose estimation accuracy. Finally, a preliminary hardware-in-the-loop campaign qualitatively validates the physical deployment of the pipeline, identifying target symmetry and accumulated tracking drift as critical factors for robust navigation. DreamSat-Bench provides a rigorous framework for maturing autonomous navigation with unprepared space assets in the absence of prior geometric models.