Meta learning improves identifiability in hybrid deep generative models
On the Identifiability of Hybrid Deep Generative Models: Meta-Learning as a Solution
Machine Learning
Summary
Sometimes, computer programs try to learn how things work by mixing known scientific rules with flexible neural networks. However, it can be hard for these programs to figure out the exact parts of the model that come from science and the parts that come from learning. The authors studied whether these hybrid models can be uniquely identified and found that many existing models cannot. They propose a new way using meta-learning that helps these models figure out the correct parts more reliably, and they tested this on both fake and real data with good results.
What this means in practice
- •For machine learning engineers: Build hybrid models that reliably separate known physics components from learned parts using meta-learning techniques.
- •For data scientists in physics: Use meta-learning based hybrid models to better infer physical parameters alongside neural features from experimental data.
Authors
Yubo Ye, Maryam Toloubidokhti, Sumeet Vadhavkar, Xiajun Jiang, Huafeng Liu, Linwei Wang
Abstract
The interest in leveraging physics-based inductive bias in deep learning has resulted in recent development of hybrid deep generative models (hybrid-DGMs) that integrate known physics-based mathematical expressions in neural generative models. To identify these hybrid-DGMs requires inferring parameters of the physics-based component along with their neural component. The identifiability of these hybrid-DGMs, however, has not yet been theoretically probed or established. How does the existing theory of the un-identifiability of general DGMs apply to hybrid-DGMs? What may be an effective approach to construct a hybrid-DGM with theoretically-proven identifiability? This paper provides the first theoretical probe into the identifiability of hybrid-DGMs, and presents meta-learning as a novel solution to construct identifiable hybrid-DGMs. On synthetic and real-data benchmarks, we provide strong empirical evidence for the un-identifiability of existing hybrid-DGMs using unconditional priors, and strong identifiability results of the presented meta-formulations of hybrid-DGMs. Code is available at https://github.com/GoForIt-YYB/Meta-Hybrid-VAE.