Shared learning method improves estimation across related conditional distributions
Multi-Task Learning of Conditional Mean Operators: applications to dynamical systems and uncertainty quantification
Machine Learning
Summary
Estimating how one thing changes depending on another is important for many areas like predicting uncertainty or understanding system dynamics. The authors noticed that when estimating these relationships separately for similar tasks, valuable shared patterns are overlooked. They proposed a method that learns common features across tasks while still adapting to each specific case, improving accuracy and efficiency. Their approach works well on tasks involving physics-based systems and uncertainty quantification.
What this means in practice
- •For uncertainty quantification teams: Enhance accuracy in uncertainty estimates across related data conditions by jointly learning shared features in conditional distributions.
- •For computational physicists: Obtain compact, physically meaningful models of complex system dynamics like plasma or Langevin processes that help identify key parameters.
Authors
Sami Chemlal, Thibaut Germain, Rémi Flamary, Vladimir R. Kostic, Karim Lounici
Abstract
Estimating conditional statistics and learning representations of a population of conditional distributions are central problems in many data-driven applications, including uncertainty quantification and dynamical systems analysis. Conditional mean operators (CMOs), a class of linear operators between function spaces, resolve these objectives by providing access to a broad class of conditional statistics. However, existing methods typically estimate each CMO independently or constrain it to prespecified function spaces, thereby preventing the exploitation of shared structure across related distributions. In this work, we posit that related CMOs share finite-dimensional input and output function spaces, and are specialized for each task with a linear operator mapping these spaces. Based on this hypothesis, we introduce MTL-CMO, a multi-task framework that jointly learns shared function spaces and task-specific operators across multiple datasets. We further introduce T-CMO, a transfer learning method that reuses the shared spaces to estimate, in closed form, the operator of a new conditional distribution. We establish statistical guarantees quantifying the benefits of jointly learning the shared function spaces. Our experiments demonstrate that learning shared function spaces improves uncertainty quantification across a broad range of conditional distributions and, when applied to Langevin and plasma dynamics, yields compact representations of complex dynamics that retain physically meaningful information and enable parameter identification.