Det-lime improves explanations for multiple marine animal detections in images
Det-LIME: Detector-Aware, Multi-Instance Local Interpretable Model-Agnostic Explanations for Automated Marine Mammal Detection
Computer Vision and Pattern RecognitionArtificial Intelligence
Summary
Detecting animals like seals in drone photos is hard, especially when there are many together. Traditional explanation tools that show why a computer thinks an animal is present often focus on just one example and give blurry results. The authors created Det-LIME, a new method that explains each detected animal separately and with clearer visuals. They showed it works better than older methods, helping scientists understand and improve how the computer detects marine mammals.
What this means in practice
- •For marine conservation teams: Interpret marine mammal detector outputs to improve monitoring accuracy and guide data collection.
- •For wildlife drone operators: Use instance-specific explanations to debug and fine-tune automated animal detection in aerial imagery.
Authors
Jiayi Zhou, David W. Johnston, Brinnae Bent
Abstract
Despite the rapid uptake of black-box object detectors in marine mammal research and monitoring, explainability techniques are rarely integrated into conservation workflows. Furthermore, most classification-oriented explainability tools are ill-suited to detection tasks involving imagery of social organisms or those with colonial life histories, as they ignore multiple detections within a scene and produce single-instance outputs that blur evidence across individuals. These methods also generate low-resolution, often biologically irrelevant visuals, limiting their utility for debugging, targeted data augmentation, and refined data collection. We proposed Det-LIME, a detector-aware, multi-instance adaptation of Local Interpretable Model-Agnostic Explanations (LIME) that produced instance-specific, box-aligned explanations by combining per-detection weighting, a proximity kernel that emphasizes regions near each box, and Intersection-over-Union-based matching to track the same instance across perturbations. We evaluated Det-LIME on aerial drone imagery for harbor seal detection, with an additional seabird case study to assess generality, and compared it with vanilla LIME, Stabilized LIME, Deterministic LIME, and gradient-based attribution methods. Using the Attribution Ratio and Max Saliency Hit Rate metrics, we showed that Det-LIME consistently improved multi-instance attribution. In practice, these higher-resolution, instance-aware explanations provide insight into model outputs and support post-processing, debugging, and actionable improvements in modeling and data collection or augmentation.