Paired rgb and sonar dataset enables accessible fish monitoring
A Paired RGB-Sonar Dataset for Fish Monitoring with Consumer Sonar
Computer Vision and Pattern Recognition
Summary
Seeing underwater can be really hard when it's dark or murky. The authors collected a big dataset that matches regular camera videos with images from an affordable type of sonar to watch koi fish. This combination helps people understand what the sonar images show since they are noisy and tough to read. The dataset includes lots of pairs of images and info about how the sonar and camera views match up. Others can use this to build better software that finds and counts fish using cheaper sonar devices.
What this means in practice
- •For environmental monitoring teams: Develop automated fish detection and counting tools using affordable consumer sonar paired with video for more accessible ecological surveys.
- •For aquaculture operators: Combine sonar and video data to better track fish health and behavior in turbid or dark underwater environments for farm management.
Authors
Mourad Derbedrosian, Michael Hobley, Pietro Perona
Abstract
Imaging sonar enables underwater observation in conditions where optical imaging is ineffective due to darkness and turbid water. However, imaging-sonar systems previously used for ecological monitoring have typically been expensive, limiting their accessibility. Consumer forward-looking sonar has the potential to make fish monitoring more accessible, but its noisy imagery makes annotation and the development of automated methods challenging. We introduce the Huntington and Caltech Koi RGB and Sonar dataset (HaCKRS), which pairs Garmin LiveScope imaging-sonar recordings with top-down RGB video of koi carp. The dataset contains more than 105,000 temporally matched frame pairs, spanning approximately 3.3 hours of recordings across four range and orientation configurations. We provide frame-wise metadata, sonar geometry and coordinate information, synchronization data for all four configurations, and planar registration for the three horizontal configurations. The release focuses on paired observations and calibration rather than task-specific fish annotations. We intend HaCKRS to support annotation and the development of fish detection, tracking, and counting methods, and investigations into the potential of consumer sonar for ecological monitoring.