Bayesian deep learning improves robustness of biomedical time-series classification

Evaluating Bayesian deep learning for robust biomedical time-series classification under distribution shift: A sleep scoring case study

Machine Learning

Summary

Machine learning models that analyze biomedical time-series data often perform poorly when the new data differs from what they were trained on. The authors studied whether Bayesian deep learning methods, which combine multiple models to make decisions, help improve accuracy and reliability in these situations. Using sleep data from rodents, they found that methods considering multiple possible solutions (multi-basin) generally work better than single-solution ones. These Bayesian methods improved performance on new, different data and helped identify some unusual or faulty data but did not detect disagreements between human experts well.

What this means in practice

  • For clinical data engineers: Improve automated biomedical time-series classification systems to remain accurate and reliable despite changes in patient populations or recording environments.
  • For biomedical device developers: Enhance robustness and uncertainty estimation in sleep monitoring devices using Bayesian model averaging techniques.

Authors

Mads Birch Sørensen, Ulrike Richter, Francois Gastambide, Michael Riis Andersen

Abstract

In biomedical time-series classification, machine learning models often suffer from deteriorating accuracy and calibration due to distribution shift. We investigate to what degree Bayesian-inspired deep learning improves robustness through Bayesian model averaging (BMA). Using rodent sleep scoring as a case study, we evaluate BMA under distribution shifts in genotype, laboratory, and species. We systematically compare nine approximate inference methods spanning single- and multi-basin approaches across two neural network backbones on in-distribution (ID) data and five out-of-distribution (OOD) datasets. In our results, multi-basin methods perform best, outperforming baselines and single-basin methods. Compared with point estimates, BMA generally improves OOD accuracy and calibration across both architectures, while yielding only modest ID gains. It also often improves selective prediction and OOD and artifact detection, but, interestingly, not inter-rater disagreement detection.
View PDFOpen arXiv