TAN Classifiers Based on Decomposable Distributions

In this paper we present several Bayesian algorithms for learning Tree Augmented Naive Bayes (TAN) models. We extend the results in Meila & Jaakkola (2000a) to TANs by proving that accepting a prior decomposable distribution over TAN’s, we can compute the exact Bayesian model averaging over TAN structures and parameters in polynomial time. Furthermore, we prove that the k-maximum a posteriori (MAP) TAN structures can also be computed in polynomial time. We use these results to correct minor errors in Meila & Jaakkola (2000a) and to construct several TAN based classifiers. We show that these classifiers provide consistently better predictions over Irvine datasets and artificially generated data than TAN based classifiers proposed in the literature.

[1]  Jesús Cerquides,et al.  Tractable Bayesian Learning of Tree Augmented Naive Bayes Classifiers , 2003 .

[2]  Tommi S. Jaakkola,et al.  Tractable Bayesian learning of tree belief networks , 2000, Stat. Comput..

[3]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[4]  Seth Pettie,et al.  An optimal minimum spanning tree algorithm , 2000, JACM.

[5]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[6]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[7]  Tom Fawcett,et al.  ROC Graphs: Notes and Practical Considerations for Data Mining Researchers , 2003 .

[8]  Pat Langley,et al.  An Analysis of Bayesian Classifiers , 1992, AAAI.

[9]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[10]  Kun-Mao Chao,et al.  Counting Spanning Trees ∗ , 2000 .

[11]  Pedro M. Domingos,et al.  On the Optimality of the Simple Bayesian Classifier under Zero-One Loss , 1997, Machine Learning.

[12]  CerquidesUbilabUBS AGBahnhofstrasse.  Applying General Bayesian Techniques to Improve Tan Induction , 1999 .

[13]  Toshihide Ibaraki,et al.  An Algorithm for Finding K Minimum Spanning Trees , 1981, SIAM J. Comput..

[14]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[15]  Michael I. Jordan,et al.  Learning with Mixtures of Trees , 2001, J. Mach. Learn. Res..

[16]  Henry Tirri,et al.  Bayes Optimal Instance-Based Learning , 1998, ECML.

[17]  Fabio Gagliardi Cozman,et al.  Generation of Random Bayesian Networks with Constraints on Induced Width , with Application to the Average Analysis of d-Connectivity , Quasi-random Sampling , and Loopy Propagation , 2003 .