Underwater object detection improves with realistic synthetic data
Towards Scaling Marine Perception with Synthetic Data
RoboticsComputer Vision and Pattern Recognition
Summary
Underwater robots and cameras need lots of labeled pictures to learn how to spot things like sea urchins, but getting real underwater pictures with labels is expensive and hard. The authors made a tool that creates very realistic fake underwater scenes, automatically labeling everything so they don’t have to do it by hand. They tested this tool by training models to find sea urchins in real underwater photos and studied how changing the fake scenes affected performance. Their work helps people build better underwater perception systems without relying only on hard-to-get real data.
What this means in practice
- •For marine robotics teams: Train underwater perception models with richly labeled synthetic datasets to detect marine life and objects more efficiently.
- •For computer vision developers: Create customizable underwater scenes for training and testing vision algorithms without extensive real data collection.
Authors
Haoyu Ma, Onur Bagoren, Anja Sheppard, Elias Fandi, Ashrith Edukulla, Tanner Aslan, Natasha Sieh, Jingyu Song, Katherine A. Skinner
Abstract
Scalable machine learning in challenging underwater environments is strongly limited by the lack of labeled real-world training data. This data is often expensive and laborious to gather, making large-scale real-world data challenging to gather and curate. However, simulated data can help close the gap, enabling many learning-based tasks for underwater perception. In this work, we extend OceanSim, an IsaacSim-based underwater perception simulator, with a Synthetic Data Generation (SDG) pipeline for training models to be used in underwater scenarios. The proposed pipeline enables users to generate large, automatically labeled, photorealistic datasets with configurable scene appearance, structure, and sensor settings. We evaluate the pipeline on a real-world sea urchin detection task and study how different forms of synthetic scene variation affect sim-to-real performance. Based on these experiments, we discuss findings on our results, main limitations of the current pipeline and identify future directions for improving underwater rendering fidelity, scene diversity, and the evaluation of sim-to-real generalization. The open-source code can be found at https://github.com/umfieldrobotics/OceanSim.