Underwater vision language model improves spatial reasoning with geometry data

USI-Brain: Enhancing Underwater Spatial Intelligence via Spatial Data-Driven Vision-Language Modeling

Computer Vision and Pattern Recognition

Summary

Underwater images are hard to understand because water blurs and distorts shapes and colors, making it tough to tell where objects really are. The authors created a large set of questions and answers that connect what things look like in underwater photos with their 3D shapes to help computers learn better. They trained a model called USI-Brain that uses this information to understand underwater scenes more accurately, especially after adding a special correction step. This model scored better than previous ones on tests measuring how well it interprets underwater spaces and objects.

What this means in practice

  • For marine robotics teams: Improve underwater robots' ability to identify and locate objects despite low visibility by integrating geometry-based visual corrections.
  • For autonomous underwater vehicle developers: Enable AUVs to better interpret spatial relationships in their environments using a vision-language model trained on a large underwater QA dataset.

Authors

Zhiliang Zhu, Zhiqiang Zhao, Xie'an Jiang, Yingxin Lai, Kangye Zheng, Dawei Zhao

Abstract

Underwater spatial reasoning depends on object boundaries and geometric cues that are obscured by scattering, color distortion, and low visibility. Existing underwater resources capture these cues through images, annotations, and depth maps, but provide fragmented supervision for vision-language models. We present a spatial data engine that connects visual appearance with object-level geometry through enhancement, 2D and geometry-assisted question generation, and expert filtering. It produces 4.27M QA pairs across five capability groups. We use this corpus to train USI-Brain and introduce an Underwater Prior-guided Rectifier (UPR) that applies gated corrections to visual features. USI-Bench evaluates the resulting capabilities with 6,525 held-out QA pairs. On its 15 task metrics, USI-Brain and USI-Brain + UPR improve over Qwen3-VL-8B-Instruct by 5.76 and 7.63 points on average, respectively, using signed differences on 100-point scales. Ablations show that geometry-assisted supervision improves depth and ordering by 4.8 and 6.1 points with UPR disabled, while UPR adds 18.0 points in scene description under matched training data. Both variants use raw images at evaluation. Across NautData, UWBench, and MarineEval, USI-Brain achieves the best or joint-best score on 28 of 32 metrics. Model weights and USI-Bench will be released at https://github.com/SKY-Zhu-91/USI-Brain.