AI summaryⓘ
The authors studied common methods used to shrink data into simple pictures, like UMAP and t-SNE, which can give different pictures each time they run. They found that some sets of data can look very different in these pictures from one run to another, while other sets can look the same but might be showing random noise instead of real patterns. To fix this, the authors suggest combining many runs by looking at the 'middle' picture from them, and also slightly changing the data before making these combined pictures to reduce overfitting noise. Their tests show that these combined pictures are about as good as single runs for quality and better at showing how reliable the patterns are. They used these ideas to help people better understand when the pictures of data are trustworthy or not.
non-linear dimensionality reductionUMAPt-SNEstochastic methodsprojection variabilityensemble methodsdata perturbationconsensus embeddingoverfittingdata visualization
Authors
Kai Nylund, Michael Correll, Lace Padilla
Abstract
Widely used non-linear dimensionality reduction (NLDR) methods such as UMAP and t-SNE are stochastic--repeated runs on the same data can produce different low-dimensional projections. In this paper, we explore two problems related to projection variability: on some datasets clusters, structure, and outliers may change run-to-run, and on others projections can be extremely stable when overfitting noise. To address the first problem, we propose visualizing the median of multiple NLDR outputs rather than relying on individual projections. To address the second, we perturb input data before creating consensus embeddings. We find that taking the median of multiple projections performs comparably to individual runs on multiple quality metrics, while increasing perturbation emphasizes global over local structure. We show through a set of exploratory visualizations that even relatively simple ensemble presentations can be used to better communicate the reliability of projection patterns.