Optimizing combined deep learning models using distributional methods
A Distributional Optimisation Perspective on Combining Models in Deep Learning
Machine Learning
Summary
Combining different models can make machine learning better, but usually the models and how they are combined are chosen separately and without a clear plan. The authors look at a mathematical way called distributional optimization to jointly train both the models and their combination rules. They study two common ways of combining models and find differences in how well optimization methods work for each. They also test new algorithms on simple tasks and on improving large language models’ performance on common sense reasoning.
What this means in practice
- •For machine learning engineers: Jointly train model ensembles and their combination rules more reliably to boost prediction performance on classification tasks.
- •For nlp model fine-tuning teams: Improve fine-tuning of large language models on reasoning benchmarks by applying distributional optimization to combine adaptation strategies.
Authors
Congye Wang, Yan Lin, Zheyang Shen, Matthew A. Fisher, Chris. J. Oates
Abstract
Combining predictions from different models can improve performance at machine learning tasks, but the training of the individual models and the rule used to combine them are typically chosen separately, and by ad hoc means. Recent advances in distributional optimisation (i.e. where the optimisation occurs over the set of probability distributions) offer an opportunity for principled joint training, viewing the collection of models as a discrete distribution whose support points are to be optimised, but the potential of these methods is not well-understood. In this paper we (1) cast two standard combination strategies - ensembles and low-rank adapter averaging - as entropy-regularised distributional optimisation, observing that the resulting objective is convex in the ensemble case but not in the adapter-averaging case, so that existing convergence guarantees for mean field Langevin dynamics transfer only to the former; (2) assess existing and novel algorithms for this task, including a functional variant of variational gradient descent; and (3) report an empirical study spanning synthetic classification tasks and fine-tuning of large language models on a commonsense reasoning benchmark.