Segment guided feature density improves anomaly detection in images
SAFe: Segment-guided Aggregation of Feature Densities for Anomaly-aware Segmentation
Computer Vision and Pattern Recognition
Summary
Visual systems that break scenes into parts can struggle when they see something new or unusual. The authors propose SAFe, a method that uses advanced image features and density estimates to spot unusual objects in images better. They combine features capturing overall meaning with details from smaller image parts and group likelihoods into clear segments. This approach helps identify unexpected objects more reliably and can tell different new objects apart without extra training. Their method showed better results than previous techniques on several benchmark tests.
What this means in practice
- •For autonomous vehicle engineers: Improve perception systems to detect and segment unexpected obstacles in driving environments using combined feature density and segmentation methods.
- •For robotics perception teams: Enhance robots' scene understanding by detecting and grouping novel objects without retraining, improving navigation and interaction in unfamiliar settings.
Authors
Anja Delić, Jurica Runtas, Marin Oršić, Ivan Marković, Ivan Petrović
Abstract
Visual segmentation systems encounter objects outside their training distribution during real-world deployment, hindering reliable autonomous systems that depend on scene parsing in the perception stage. Many recent methods address this by using self-supervised foundation models to train density estimators that yield low likelihood in anomalous image regions. Although promising, these methods suffer from poor feature semantics or they lack spatial consistency, both of which undermine critical downstream decisions. We address this problem with~\method, a generative method based on class-conditional density estimation over self-supervised representations. SAFe trains lightweight normalizing flows that produce class-conditional normalized likelihood estimates over frozen DINOv3 features. We combine density estimates from transformer features with density scores over multi-scale convolutional features to capture both global semantics and local detail. We introduce a method-agnostic post-processing step based on SAM3 that connects per-location likelihoods into spatially coherent segments while suppressing false positives, and enables instance-level anomaly detection without retraining. The post processing further distinguishes novel categories among anomalous objects by a similarity-based agglomerative clustering scheme. SAFe sets a new state of the art on the PANIC, OoDIS, SMIYC ObstacleTrack with strong performance on the ISSU benchmark.