Vision foundation model improves spacecraft pose estimate accuracy
Vision Foundation Models with Synthetic-Only Training for Monocular Spacecraft Pose Estimation
Computer Vision and Pattern RecognitionRobotics
Summary
Estimating the position and orientation of spacecraft from a single camera image is important but challenging. The authors improved this task by using a much larger and more advanced vision model than before, which was trained only on simulated images. Their approach achieved more accurate results than previous methods on standard test environments that mimic real spacecraft lighting conditions. This shows that powerful models trained on just synthetic data can work well for important space applications.
What this means in practice
- •For satellite operators: Use improved pose estimates from single images to aid spacecraft navigation and inspection without relying on cooperative targets or extensive real image training data.
- •For embedded system engineers: Deploy high-accuracy vision-based pose estimation on compact, power-efficient processors with proven space flight heritage for real-time onboard spacecraft operations.
Authors
John Church, Vazghen Nikolian
Abstract
We present an improvement on previous spacecraft pose estimation architectures that results in the lowest published mean rotation errors we know of on the SPEED+ lightbox and sunlamp test sets for a known, non-cooperative spacecraft. By using a previously established heatmap-based pose estimation architecture and adapting a large self-supervised ViT foundation model (DINOv3) in place of the smaller convolutional and ViT encoders of previous work, we show that pose estimation accuracy improves from 300M to 840M parameters with no saturation yet observed. We also evaluate our 840M model on a Jetson Orin NX 16GB, measuring single-pass network inference at 133.8 ms per crop with a board draw of 32.0 W. These measurements demonstrate embedded inference feasibility on a processor family with orbital flight heritage. Our resulting model outperforms previous models across lightbox and sunlamp domains while training only on synthetic data. Our best model, using DINOv3 840M adapted with LoRA as the encoder (rank 64, three-seed ensemble with four-rotation test-time augmentation), results in $1.56^\circ$ mean rotation error on sunlamp and $1.17^\circ$ on lightbox, compared to the previous best mean rotation errors we know of on these test sets, $2.66^\circ$ and $1.75^\circ$ by EagerNet.