Learning Robust Bayesian Network Classifiers in the Space of Markov Equivalent Classes

Tree Augmented Na¨ýve Bayes(TAN) is a robust classification model. However, so far some researchers still attempt to improve the performance by considering directions of edges, because traditional learning method merely takes into account log likelihood, which is not suitable for learning classifiers, when learning a tree topological structure. In this paper, we analyze search spaces of TAN, research equivalent classes in them. Accordingly, we point out it is not necessary to pay attention to the dependent directions between conditional variables for these directions do not play a role in maximizing log conditional likelihood. For application, we propose a novel framework for learning TAN classifiers. Finally, we run experiments on Weka platform using 45 problems from the University of California at Irvine repository. Experimental results show that classification accuracy and stability do not change statistically in our leraning framework.

[1]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[2]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[3]  Wei Luo,et al.  A new hybrid method for Bayesian network learning With dependency constraints , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[4]  Luis M. de Campos,et al.  A Scoring Function for Learning Bayesian Networks based on Mutual Information and Conditional Independence Tests , 2006, J. Mach. Learn. Res..

[5]  Bin Shen,et al.  Structural Extension to Logistic Regression: Discriminative Parameter Learning of Belief Net Classifiers , 2002, Machine Learning.

[6]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[7]  Eamonn J. Keogh,et al.  Learning the Structure of Augmented Bayesian Classifiers , 2002, Int. J. Artif. Intell. Tools.

[8]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[9]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[10]  Pedro Larrañaga,et al.  Learning Bayesian network structures by searching for the best ordering with genetic algorithms , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[11]  Bertran Steinsky,et al.  Enumeration of labelled chain graphs and labelled essential directed acyclic graphs , 2003, Discret. Math..

[12]  Pedro M. Domingos,et al.  Beyond Independence: Conditions for the Optimality of the Simple Bayesian Classifier , 1996, ICML.

[13]  Luis M. de Campos,et al.  Searching for Bayesian Network Structures in the Space of Restricted Acyclic Partially Directed Graphs , 2011, J. Artif. Intell. Res..

[14]  David Maxwell Chickering,et al.  Large-Sample Learning of Bayesian Networks is NP-Hard , 2002, J. Mach. Learn. Res..

[15]  Wray L. Buntine Theory Refinement on Bayesian Networks , 1991, UAI.

[16]  S. Miyano,et al.  Finding Optimal Bayesian Network Given a Super-Structure , 2008 .

[17]  Liangxiao Jiang,et al.  Learning Tree Augmented Naive Bayes for Ranking , 2005, DASFAA.

[18]  Remco R. Bouckaert,et al.  Probalistic Network Construction Using the Minimum Description Length Principle , 1993, ECSQARU.

[19]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[20]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.