Virtual sensing robustness tested across failures and domains
Reliable Virtual Sensing: A Multi-Domain Benchmark for Robustness Under Sensor Failures
Machine LearningArtificial Intelligence
Summary
Measuring some important quantities in systems can be hard because sensors sometimes fail. The authors created a new benchmark called MuViS-C that tests how well computer models predict under ten types of sensor failures. They tested many models across different fields and found all models struggle with sensor problems, but some are better than others. Their benchmark helps to find which methods are more reliable when sensors fail, making these systems safer and more trustworthy.
What this means in practice
- •For industrial control engineers: Evaluate and select virtual sensing models that maintain performance despite sensor failures in factory and process automation systems.
- •For autonomous vehicle developers: Test robustness of sensor-based estimators in vehicles under realistic sensor degradation or dropout scenarios to improve safety.
Authors
Jens U. Brandt, Noah C. Puetz, Alexander Windmann, Marc Hilbert, Elena Raponi, Thomas Bäck, Thomas Bartz-Beielstein
Abstract
Virtual sensing, the estimation of hard-to-measure quantities from available sensor measurements, is a critical enabler for control and monitoring in cyber-physical systems. However, when sensors fail, learning-based predictors can produce physically implausible estimates that propagate to system-level failures. We argue that real-world deployment demands robustness and introduce MuViS-C, the first multi-domain benchmark of robustness against common sensor failures in learning-based virtual sensing. Building on an existing nominal-performance benchmark and established corruption taxonomies, it covers ten sensor failure modes, from subtle drifts to catastrophic signal dropouts, at multiple severities. These are paired with complementary robustness measures capturing average error under corruption, relative degradation, and worst-case fragility. Across nine datasets from six domains, we benchmark six architectures spanning gradient-boosted trees and the major inductive biases for sequence modeling: convolution, recurrence, attention, and MLP-mixing. On the attention-based architecture, we further probe three robustification strategies. We find that (i) every model degrades substantially under corruption, becoming worse than a naïve predictor on at least one corruption setting, (ii) gradient-boosted tree ensembles achieve strong robustness, and (iii) dedicated robustification closes the gap between the attention-based architecture and the most robust models, though each strategy hurts nominal performance. The benchmark's multi-domain design proves essential, as model rankings shift across datasets, and no single domain captures the full robustness picture. MuViS-C is open-source and extensible to new datasets, failure modes, measures, and models.