Incremental Learning of Tree Augmented Naive Bayes Classifiers

Machine learning has focused a lot of attention at Bayesian classifiers in recent years. It has seen that even Naive Bayes classifier performs well in many cases, it may be improved by introducing some dependency relationships among variables (Augmented Naive Bayes). Naive Bayes is incremental in nature but, up to now, there are no incremental algorithms for learning Augmented classifiers. When data is presented in short chunks of instances, there is an obvious need for incrementally improving the performance of the classifiers as new data is available. It would be too costly, in computing time and memory space, to use the batch algorithms processing again the old data together with the new one. We present in this paper an incremental algorithm for learning Tree Augmented Naive classifiers. The algorithm rebuilds the network structure from the branch which is found to be invalidated, in some sense, by data. We will experimentally demonstrate that the heuristic is able to obtain almost optimal trees while saving computing time.

[1]  Pat Langley,et al.  Induction of Selective Bayesian Classifiers , 1994, UAI.

[2]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[3]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[4]  Wray L. Buntine.  A Guide to the Literature on Learning Probabilistic Networks from Data , 1996, IEEE Trans. Knowl. Data Eng..

[5]  David J. Spiegelhalter,et al.  Sequential updating of conditional probabilities on directed graphical structures , 1990, Networks.

[6]  Russell Greiner,et al.  Learning Bayesian Belief Network Classifiers: Algorithms and System , 2001, Canadian Conference on AI.

[7]  Eamonn J. Keogh,et al.  Learning augmented Bayesian classifiers: A comparison of distribution-based and classification-based approaches , 1999, AISTATS.

[8]  Douglas H. Fisher,et al.  Ordering Effects in Incremental Learning , 1993 .

[9]  Nir Friedman,et al.  Sequential Update of Bayesian Network Structure , 1997, UAI.

[10]  Hans Spada,et al.  Learning in Humans and Machines: Towards an Interdisciplinary Learning Science , 1995 .

[11]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[12]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[13]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[14]  Wai Lam,et al.  Using New Data to Refine a Bayesian Network , 1994, UAI.