Time series domain adaptation improved with auto thresholding method

A Comprehensive Benchmark of Source-Free Universal Domain Adaptation on Time Series Representations

Machine Learning

Summary

Adapting computer models to work well on new types of time series data without accessing original training data is hard, especially when new data may have unseen categories. The authors created the first benchmark testing methods for this problem in time series data. They found that pretrained models don’t always perform better than older methods in this setting. They also discovered that deciding when to reject unknown categories is very sensitive and proposed an automatic technique to improve this decision. Their experiments on several datasets show this new approach works well and highlight the need for better models tailored to time series.

What this means in practice

  • For time series engineers: Improve adaptation of models to new unlabeled time series with unknown categories using an auto-threshold method for better unknown sample detection.
  • For speech recognition developers: Use insights about threshold sensitivity and feature extractors to adapt speech models to new environments without source data access.

Authors

Romain Mussard, Fannia Pacheco, Maxime Berar, Paul Honeine, Gilles Gasso

Abstract

Source-Free Universal Domain Adaptation (SF-UniDA) extends Universal Domain Adaptation by removing access to source data at adaptation time while still handling label-set mismatches between domains. Despite growing interest in this setting for image data, no benchmark exists for time series, which are more challenging. We present the first SF-UniDA benchmark on time series. In addition, we provide the first study of pretrained foundation models as feature extractors for time series domain adaptation. In this context, we identify a critical and previously underexplored limitation of all existing SF-UniDA methods: the inference threshold for unknown-sample rejection is highly sensitive. We address this by proposing a plug-in auto-thresholding module that can be integrated into any SF-UniDA method. Experiments on three well-known time series datasets confirm the suitability of this module. They also highlight that foundation models do not systematically outperform classical backbones and that SF-UniDA tailored for time series is yet to be developed.
View PDFOpen arXiv