Visualizing feature importance with weighted t-SNE boosts data understanding
Assessing feature scorer results on high-dimensional datasets with t-SNE
Machine LearningArtificial Intelligence
Summary
When datasets have lots of features, it can be hard to tell which ones matter most. The authors introduce a way to show the importance scores of features visually by changing how data points are displayed in a 2D space. This helps users compare different scoring methods and see how models organize data based on important features. Their method works on both fake and real datasets and can make machine learning models easier to understand.
What this means in practice
- •For machine learning engineers: Compare and select feature scoring methods by visualizing their impacts on data representation to improve model interpretability.
- •For data analysts: Visually identify which features drive patterns in complex tabular datasets to guide feature selection and knowledge discovery.
Authors
Bruno Iochins Grisci, Mario Inostroza-Ponta, Márcio Dorn
Abstract
While vast literature on high-dimensional data visualization is available, there are not many works regarding the visualization of feature scorers and their results. Feature scorers are algorithms that assign numerical importance to each feature of multi-dimensional datasets. These importance scores can be used in several applications, such as feature selection, knowledge discovery, and machine learning interpretability. There are several feature scorers to choose from, and often no single metric or ground truth is available to guarantee the quality of their results. In this scenario, visualization can become valuable to support the decision of which method to choose and how good its results are. For this goal, this work presents "weighted t-SNE." It modifies the relationship between data points in the embedded 2D space to reflect the importance of each dimension of the original datasets as assessed by a feature scorer. This research discusses how to implement weighted t-SNE, proposes the silhouette coefficient as a numerical evaluation of the results, and shows several examples of its use in practice. Synthetic and real-world tabular datasets are used in the experiments together with nine feature scorers, ranging from Mutual Information to neural networks. Each feature scorer produces unique visualizations, and weighted t-SNE can be used to compare and choose the one that better suits a given dataset and task. Weighted t-SNE can also visually show the importance of features learned by machine learning models and help us see how they are organizing the data, increasing their interpretability.