Semi-hierarchical naïve Bayes classifier

The classification of high dimensional data is an arduous task especially with the emergence of high quality data acquisition techniques. This problem is accentuated when the whole set of features is needed to learn a classifier such as the case of genomic data. The Bayesian approach is suitable for these applications because it represents graphically and statistically the dependencies between the features. Unfortunately, learning a Bayesian classifier using a high number of features does not ensure a tradeoff between the dimensions' reduction, the semantic of the model and the predictive performance. We propose a new semi-hierarchical naïve Bayes that uses the latent variables for abstracting the features of a given dataset in order to reduce the dimensionality. These variables are suitable for finding graphically and semantically analyzable models. We combined them with the observed variables in a tree-augmented naïve Bayes structure in order to improve the prediction accuracy. An excessive experimental study showed that our method is suitable for high dimensional data and overcomes the existing methods.

[1]  Walid Mahdi,et al.  A New Equilibrium Criterion for Learning the Cardinality of Latent Variables , 2015, 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI).

[2]  Kevin P. Murphy,et al.  Learning the Structure of Dynamic Probabilistic Networks , 1998, UAI.

[3]  Chenxi Liu.  Bayes Decision Theory , 2009, Encyclopedia of Biometrics.

[4]  Tao Chen,et al.  Latent Tree Classifier , 2011, ECSQARU.

[5]  Ron Kohavi,et al.  Supervised and Unsupervised Discretization of Continuous Features , 1995, ICML.

[6]  Thomas D. Nielsen,et al.  Classification using Hierarchical Naïve Bayes models , 2006, Machine Learning.

[7]  Matthew Self,et al.  Bayesian Classification , 1988, AAAI.

[8]  S. C. Johnson.  Hierarchical clustering schemes , 1967, Psychometrika.

[9]  Thomas D. Nielsen,et al.  Latent variable discovery in classification models , 2004, Artif. Intell. Medicine.

[10]  H. Akaike.  A new look at the statistical model identification , 1974 .

[11]  Tao Chen,et al.  Efficient Model Evaluation in the Search-Based Approach to Latent Structure Discovery , 2008 .

[12]  Yi Wang,et al.  Latent tree models for multivariate density estimation: algorithms and applications , 2009 .

[13]  Marco Wiering,et al.  2011 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) , 2011, IJCNN 2011.

[14]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[15]  Frank Nielsen,et al.  On weighting clustering , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Harry Zhang,et al.  The Optimality of Naive Bayes , 2004, FLAIRS.

[17]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[18]  Nir Friedman,et al.  Learning the Dimensionality of Hidden Variables , 2001, UAI.

[19]  T. Moon.  The expectation-maximization algorithm , 1996, IEEE Signal Process. Mag..