Papers for

robotic perception engineers

Papers whose findings have a practical use for this group, as judged from the abstract. Open a paper to read what it means in practice.

Unified alignment improves robot perception across multiple sensor types

UniOMA: Unified Optimal-Transport Multi-Modal Structural Alignment for Robot Perception

Abstract: Achieving generalizable and well-aligned multimodal representations remains a core challenge in robot learning. While recent approaches have attempted to align modalities by modeling conditional or higher-order statistical dependencies, they do not explicitly consider the structural coherence across modalities. In this work, we propose a novel multimodal alignment method that augments existing contrastive losses with a geometry-aware Gromov--Wasserstein (GW) distance-based regularization. To this end, we encode intra-modal geometry for each modality and reduce cross-modal structural discrepancies through a dynamically learned GW barycenter, thereby encouraging structural consistency in addition to instance-level contrastive alignment. We apply this optimal-transport-based alignment strategy to robot perception tasks involving underexplored modalities such as force and tactile signals, where modality data often exhibit varying sample densities. Extensive experiments on five robotic benchmarks show that our method improves structural alignment and downstream performance over state-of-the-art multimodal contrastive baselines across a wide range of downstream tasks, including robot and environment state prediction. Moreover, our GW-based regularization can be seamlessly integrated into most InfoNCE-like objectives.

Mon 9 NovRoboticsArtificial Intelligence
The gist
Robots often need to understand information that comes from very different sensors, like touch or force sensors, but making sense of all these signals together is tricky. The authors designed a new way to better align these different types of sensor data by looking not just at individual matching parts, but at the overall structure and relationships in the data. Their method helps robots make sense of complex environments more reliably by combining data in a more organized way. They tested their approach on several robot tasks and found it works better than previous methods.
Open → 2611.12351v1

Geodesic eccentricity helps detect 3D keypoints on objects

GEKey: Learning Geodesic Eccentric 3D Keypoints via Self-Supervision

Abstract: Point clouds represent 3D object geometry as sets of spatial coordinates. A fundamental task in their analysis is keypoint detection, i.e., identifying a sparse subset of points that captures the most salient features. While supervised methods rely on manual annotations, unsupervised approaches leverage intrinsic geometric properties to localize keypoints without human input. We introduce GEKey, a self-supervised framework that uses the geodesic eccentricity transform, an SE(3)-invariant geometric measure, to identify instance-specific salient points. These points serve as pseudo-labels to train a model that aligns its predictions with the object's intrinsic structure. GEKey is simple, efficient, and robust, preserving intra-class variation by balancing instance-specific and category-level structure. Extensive experiments on ShapeNetCoreV2, KeypointNet, CAPE, ITOP, and DeformingThings4D show state-of-the-art performance in both keypoint detection and alignment, with strong resilience to noise, downsampling, deformations, and real-world conditions. The code and pretrained models are available at https://github.com/Pierreoo/GEKey.

Thu 5 NovComputer Vision and Pattern Recognition
The gist
Detecting important points on 3D objects helps computers understand shapes. The authors created GEKey, a method that finds special points on objects without needing human labels. It uses a geometric property called geodesic eccentricity to find points that stand out for each object. Their method works well even when objects are noisy, incomplete, or change shape.
Open → 2611.07677v1

Image quality alone does not predict sonar detection success

Beyond Pixel Similarity: Task-Aware Evaluation of GAN-Based Synthetic Sonar Data for Robotic Perception

Abstract: Synthetic data can reduce the cost of collecting and annotating training data for robotic perception, but generating sensor observations that preserve the characteristics relevant to downstream perception remains challenging, particularly for sonar imagery. In this work, we investigate whether conventional image-fidelity metrics adequately reflect the downstream perception performance of GAN-generated synthetic sonar data. We employ a Pix2Pix conditional generative adversarial network with four discriminator configurations characterized by different receptive fields: PixelGAN, PatchGAN-16, PatchGAN-70, and ImageGAN. The models are trained using sonar imagery from two datasets and evaluated using conventional image-fidelity metrics, including Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Mean Squared Error (MSE). To complement these pixel-level measures with task-oriented evaluation, YOLOX-S, YOLOX-L, and Faster R-CNN detectors are trained exclusively on real sonar imagery and subsequently evaluated on the GAN-generated images using identical test samples and annotations across all discriminator configurations. The results reveal a discrepancy between image-fidelity and downstream object-detection performance: the configuration achieving the best SSIM, PSNR, and MSE does not consistently yield the best detection performance. In particular, PatchGAN configurations achieve strong downstream detection results despite not achieving the highest pixel-level similarity scores. These findings suggest, for the datasets and models considered, pixel-level image-fidelity metrics alone may not consistently capture the task-relevant realism of synthetic sonar observations and motivate the use of task-aware evaluation for synthetic sensor data intended for robotic perception.

Wed 16 SeptRoboticsComputer Vision and Pattern RecognitionMachine Learning
The gist
Generating fake sonar images for training robots can save time and money, but making these images look real doesn’t always make them useful for spotting objects. The authors show that common measures of image quality don’t always match how well object detectors work on those images. They tested different ways of making fake sonar pictures and found some that help detection even though they don’t score highest on image quality tests. This means it’s important to check synthetic images by how well they help with tasks, not just by how good they look.
Open → 2609.18100v1