Marine vehicle safety improved by combining vision and sensors in low visibility
Navigational Safety of Marine Surface Vehicles in Restricted Visibility: From Intelligent Vision to Vision-Centered Multi-Sensor Fusion
Computer Vision and Pattern Recognition
Summary
Marine surface vehicles often rely on cameras to spot dangers, but bad weather and shaky motion can make images unclear. The authors review ways to improve what these vehicles 'see' by enhancing videos and using other sensors like radar and GPS together with cameras. Combining different sensors helps ships navigate safely even when visibility is poor. They also provide datasets and suggest future research to keep improving smart ship navigation.
What this means in practice
- •For marine navigation teams: Improve situational awareness on ships by integrating visual cameras with radar, AIS, and LiDAR sensors for safer navigation in poor visibility.
- •For autonomous vehicle developers: Design smarter marine autonomous systems that combine various sensor data to better detect hazards and navigate restricted-visibility waters.
A survey. It maps existing work.
Authors
Ryan Wen Liu, Fengchuan Song, Tianxiang Wang, Guobei Peng, Youngrong Kim, Xian Zhong, Hooi-Siang Kang, Maohan Liang, Junzhi Yu, Shanliang Yao
Abstract
Marine surface vehicles (MSVs) have attracted considerable attention for their commercial, scientific, and military applications. Owing to their favorable performance-to-cost ratio, shipboard cameras are widely used to detect and recognize potentially hazardous situations across diverse operational scenarios. However, visual perception is inevitably degraded by restricted conditions, including irregular motion of the imaging platform, adverse weather, and long imaging distances. To improve situational awareness and navigation capabilities, complementary sensors, such as the automatic identification system (AIS), radar, and light detection and ranging (LiDAR), etc., have become essential shipboard equipment. Multi-sensor fusion can provide richer perceptual information and help ensure navigational safety of MSVs in restricted visibility. This article reviews recent progress from intelligent vision to vision-centered multi-sensor fusion for newcomers and researchers in intelligent ship navigation and intelligent waterborne transportation system (IWTS). After introducing the generation of quality-degraded images in restricted visibility, we will review intelligent vision methods for four key issues, i.e., shipboard video stabilization, visual perception enhancement, object detection and tracking, and navigable area segmentation. We then examine vision-centered multi-sensor fusion methods, including infrared-visible image fusion, radar-vision fusion, LiDAR-vision fusion, AIS-vision fusion, and other multi-sensor fusion scenarios. Open-source datasets and safety-critical applications are also summarized and discussed. Finally, we highlight open challenges and future research directions. The collected maritime datasets are available at https://github.com/MAPGroup/Datasets and will be regularly updated.