Convolutional dropout improves classification and uncertainty in astronomical spectra
Bayesian classification of astronomical spectra with class uncertainties
Machine Learning
Summary
Classifying types of stars and galaxies from their light spectra is tricky, especially when data is unclear or noisy. The authors compared several approaches to predict these classes and express how uncertain these predictions are. They found that using a convolutional neural network with a technique called Monte Carlo dropout gave the best balance between accuracy, speed, and trustworthy uncertainty estimates. This means it not only guesses the categories well but also knows when it might be unsure.
What this means in practice
- •For astronomical survey teams: Improve classification and uncertainty estimates of stellar and galaxy spectra for large-scale sky surveys using Monte Carlo dropout enhanced CNNs.
- •For remote sensing analysts: Integrate uncertainty-aware CNN classifiers to better label spectral data from Earth observation instruments with reliable confidence measures.
Authors
Simon Barton, Martin Sahlén, Andreas Korn, Christian Glaser
Abstract
Context: We developed a probabilistic machine learning method with the aim of performing the O(10)-way classification of low- and high-resolution spectra of stellar and extragalactic targets for the upcoming 4MOST survey. In fulfilment of the survey requirements, this method should be able to express uncertainty in the input data as well as uncertainty introduced in its prediction. Aims: Four different methods are explored: (1) convolutional neural networks (CNNs), (2) the Dirichlet distribution, (3) Monte Carlo dropout (MCD), (4) Bayesian neural Networks (BNNs) + variational inference (VI). Training and validation was performed using labelled spectra from the SDSS database and a custom 4MOST mock dataset. All the methods were compared in terms of the same metrics: accuracy, area under the curve (AUC), expected calibration error (ECE), Shannon entropy, negative log-likelihood (NLL), Brier score, training time, and inference time. Methods: A CNN with simple architecture and about 20,000 parameters was trained to achieve classification accuracies of 91.5% on SDSS data and 92.8% on 4MOST mock data. The direct Dirichlet prediction and VI models tested provide uncertainties on class membership probabilities, but they confuse classes more often. The MCD on a CNN is found to be the most suitable; it boosts the point-estimate accuracies to 92.6% and 93.9%, while still providing fast training and sufficiently fast inference. Compared to a standard CNN, the method additionally provides well-calibrated uncertainties at marginal extra cost.