Generalized naive bayes extended for continuous variables with efficient structure learning
On Generalized Naive Bayes with Continuous Features
Machine Learning
Summary
The paper deals with improving a popular method called Generalized Naive Bayes (GNB), which was originally made for data categories, so it can also work well with continuous numbers. The authors found that learning the model’s structure depends on simple two-variable relationships, and they showed how to use a math tool called a matroid for efficient model building. They also present more flexible approaches for handling different kinds of data distributions and introduce a method to simplify the model for easier use. Finally, they compare their improved GNB with other understandable classification methods on real data.
What this means in practice
- •For data scientists: Build more accurate and interpretable classification models when data contains continuous variables by using efficient greedy structure learning based on pairwise dependencies.
- •For machine learning engineers: Implement GNB classifiers that adapt to various continuous data distributions with flexible copula models and apply model reduction techniques to maintain interpretability.
Authors
Ábrahám Papp, Botond Szilágyi, Edith Alice Kovács
Abstract
The Generalized Naive Bayes (GNB) model was introduced for discrete and categorical random variables as an extension of classic Naive Bayes. We now accommodate the GNB framework to continuous explanatory variables. A central result of the paper is that structure learning of the GNB depends only on the pair copulas of the bi-variate marginals. We proved that the GNB structure can be assigned to the basis of a matroid, therefore we give greedy algorithms for finding the optimal GNB structure on the training data, in sense of minimizing Kullback-Leibler divergence. Three cases are considered: joint Gaussian distribution, then a more flexible model where we suppose the dependence structure to be described by a Gaussian copula with arbitrary marginals, and an even more flexible case where the joint continuous probability distribution is arbitrary, i.e. copula and marginal distributions are arbitrary. A method for model reduction, based on the newly introduced concept of GNB forest is given. We close the paper by comparing the newly introduced GNB classification results to other classical "glass-box" algorithms on real datasets.