Experimental study on generalization capability of extended naive Bayesian classifier

Extended naive Bayesian classifier (ENBC) is a general framework of NBCs, which is developed based on $$t$$t-norm based ordered weighted averaging ($$t$$t-OWA) operator and uses the weighted summation of products of margin probabilities to determine class-conditional probability. Since ENBC was proposed in 2006, there is no such a study which tests the performances of ENBC on the real classification datasets. Thus, in this paper we conduct an experimental investigation to ENBC’s generalization capability based on 44 benchmark KEEL and UCI datasets. The analysis shows that (1) ENBC is instable and its aggregation weights are sensitive to the order of training samples and (2) ENBC indeed has higher generalization capability than the existing NBCs, e.g., normal naive Bayesian and flexible naive Bayesian when its weight is properly selected.

[1]  Emilie Philippot,et al.  Bayesian networks for incomplete data analysis in form processing , 2014, International Journal of Machine Learning and Cybernetics.

[2]  Liangxiao Jiang,et al.  A Novel Bayes Model: Hidden Naive Bayes , 2009, IEEE Transactions on Knowledge and Data Engineering.

[3]  Geoffrey I. Webb,et al.  Learning by extrapolation from marginal to full-multivariate probability distributions: decreasingly naive Bayesian classification , 2011, Machine Learning.

[4]  Pedro Larrañaga,et al.  Bayesian classifiers based on kernel density estimation: Flexible classifiers , 2009, Int. J. Approx. Reason..

[5]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[6]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[7]  Mark A. Hall,et al.  A decision tree-based attribute weighting filter for naive Bayes , 2006, Knowl. Based Syst..

[8]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[9]  Ronald R. Yager,et al.  Extending multicriteria decision making by mixing t‐norms and OWA operators , 2005, Int. J. Intell. Syst..

[10]  Yung C. Shin,et al.  A variational Bayesian framework for group feature selection , 2012, International Journal of Machine Learning and Cybernetics.

[11]  Ronald R. Yager,et al.  On ordered weighted averaging aggregation operators in multicriteria decisionmaking , 1988, IEEE Trans. Syst. Man Cybern..

[12]  Ronald R. Yager,et al.  An extension of the naive Bayesian classifier , 2006, Inf. Sci..

[13]  Geoffrey I. Webb,et al.  Not So Naive Bayes: Aggregating One-Dependence Estimators , 2005, Machine Learning.

[14]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[15]  Geoffrey I. Webb,et al.  Alleviating naive Bayes attribute independence assumption by attribute weighting , 2013, J. Mach. Learn. Res..

[16]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[17]  Yu-Lin He,et al.  Non-Naive Bayesian Classifiers for Classification Problems With Continuous Attributes , 2014, IEEE Transactions on Cybernetics.

[18]  Zhimin Chen,et al.  An intuitionistic fuzzy weighted OWA operator and its application , 2013, Int. J. Mach. Learn. Cybern..

[19]  Jing Li,et al.  A Sparse Structure Learning Algorithm for Gaussian Bayesian Network Identification from High-Dimensional Data , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Ronald R. Yager,et al.  On ordered weighted averaging aggregation operators in multicriteria decision-making , 1988 .