Vision tactile sensor detects object rotation accurately in varying gravity
A Minimal Optical-Flow Representation for Vision-Based Tactile Rotation Classification in Robotic Manipulation Across Gravity Domains
RoboticsComputer Vision and Pattern Recognition
Summary
Tactile sensors that use vision can tell how objects rotate when touched, but their detailed image data is heavy to process, especially in space robots with limited computing power. The authors studied a smaller set of motion features derived from optical flow in these sensors to classify object rotation across different gravity levels—Earth, Mars, Moon, and orbit. They found that gravity changes the sensor features slightly, enough to hurt accuracy if the model only trains on Earth data. Training on multiple gravity environments fixes this, making tactile rotation detection reliable across gravity levels.
What this means in practice
- •For space robotics engineers: Improve tactile-based object handling in robots operating under different gravity conditions by using compact optical-flow features for accurate rotation classification.
- •For industrial robot developers: Implement efficient tactile rotation detection on resource-constrained robots to maintain high accuracy without heavy image processing overhead.
Authors
Oscar Martinez-Bernal, Mario Cavero-Vidal, Francesco Grella, Carol Martinez
Abstract
Vision-based tactile sensors provide rich contact information, but processing high-resolution images can be costly for resource-constrained platforms such as space robots. This work investigates whether a compact representation of tactile motion can classify object rotation across different gravity conditions. Dense optical flow from a simulated GelSight Mini is aggregated over a 7x9 grid into 126 features and used to classify the direction of load-induced rotation under Earth, Mars, Moon, and orbital gravity. Gravity causes a small but significant shift in these features, accounting for 1.6% of their variance (R2 = 0.016). Despite its small magnitude, this shift affects models trained only on Earth data: XGBoost accuracy decreases from 94.4% on Earth to 75.9% in orbit. In contrast, a single model trained across all four gravity domains achieves 96.3% overall accuracy and 95.1%-97.0% across individual domains, without using gravity as an input. The representation can also be reduced to 40 features while retaining 95.7% accuracy, with XGBoost requiring only 0.14 ms per inference. These findings show that Earth-gravity performance alone is insufficient to establish the transferability of tactile perception for space robotic manipulation, highlighting the need to account for gravity-induced domain shifts during training and validation.