Comparing early and late RGB infrared fusion for better UAV spotting
Early versus Late RGB-Infrared Fusion for Counter-UAV Detection: A Comparative Study
Computer Vision and Pattern Recognition
Summary
Detecting small flying drones is hard, especially in bad weather or tricky lighting. The authors studied two ways to combine regular color images with infrared images to spot drones better. One method mixes these images right away before searching for drones, while the other mixes the drone detections found separately from each camera. They found that mixing images early works better overall, but mixing results later is helpful when cameras don’t line up perfectly. They also explain practical tips and trade-offs for using these methods in real-life drone detection.
What this means in practice
- •For drone security teams: Improve drone detection in challenging environmental conditions by choosing the best fusion method for combining visible and infrared sensors.
- •For surveillance system developers: Adapt drone detection workflows to handle misaligned sensors by using decision-level fusion to increase reliability in real-world deployments.
Authors
Alberto De Zan, Denis Tavaris, Gian Luca Foresti, Ivan Scagnetto
Abstract
This paper presents a novel strategy for UAV detection through visible (RGB) and infrared (IR) fusion, in order to improve accuracy and precision in harsh environments and weather conditions. In this research, two opposite approaches have been studied and implemented: the first concerns image-level (early) fusion, which merges two different spectra into a single image to which one detector is applied. On the other hand, the second approach, called decision-level (late) fusion, focuses on blending together the output bounding boxes of two different detection pipelines, aiming to create a common result that can be applied to both spectra simultaneously. Both strategies have been extensively tested in a dedicated environment, which included alternating low and high brightness scenes, different backgrounds (sky and ground), distances (5, 25 and 60 m), altitudes and zoom levels (1X, 2X and 4X). These tests allowed for the assessment of actual detection capabilities and system reliability in real-world applications. Furthermore, the two fusion strategies were evaluated and compared over a set of aligned images in order to highlight the better approach of the two. The results indicate that early fusion achieves better raw performance, since it incorporates the thermal information from the beginning of the training process. However, late fusion presents an innovative and useful detection workflow that can be applied in cases of sensor or frame misalignment. Both positive and negative aspects have also been pointed out and discussed, in order to give a more comprehensive overview. Finally, the paper outlines implementation considerations and practical recommendations based on operational priorities. Overall, the findings contribute to the research in RGB-IR fusion modalities, with the goal of enhancing UAV detection capabilities.