Building Bayesian network classifiers through a Bayesian complexity monitoring system

Abstract Nowadays, the need for practical yet efficient machine learning techniques for engineering applications are highly in demand. A new learning method for building Bayesian network classifiers is presented in this article. The proposed method augments the naive Bayesian (NB) classifier by using the Chow and Liu tree construction method, but introducing a Bayesian approach to control the accuracy and complexity of the resulting network, which yields simple structures that are not necessarily a spanning tree. Experiments by using benchmark data sets show that the number of augmenting edges by using the proposed learning method depends on the number of training data used. The classification accuracy was better, or at least equal, to the NB and the tree augmented NB models when tested on 10 benchmark data sets. The evaluation on a real industrial application showed that the simple Bayesian network classifier outperformed the C4.5 and the random forest algorithms and achieved competitive results against C5.0 and a neural network.

[1]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[2]  Mohamed Othman,et al.  A Naïve-Bayes classifier for damage detection in engineering materials , 2007 .

[3]  David Maxwell Chickering,et al.  Learning Bayesian Networks is , 1994 .

[4]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[5]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[6]  Luis M. de Campos,et al.  Learning Bayesian Network Classifiers: Searching in a Space of Partially Directed Acyclic Graphs , 2005, Machine Learning.

[7]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[8]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[9]  Eamonn J. Keogh,et al.  Learning the Structure of Augmented Bayesian Classifiers , 2002, Int. J. Artif. Intell. Tools.

[10]  Robert Biernacki,et al.  Modeling of manufacturing processes by learning systems: The naïve Bayesian classifier versus artificial neural networks , 2005 .

[11]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[12]  Martin E. Hellman,et al.  Probability of error, equivocation, and the Chernoff bound , 1970, IEEE Trans. Inf. Theory.

[13]  Byoung-Tak Zhang,et al.  Bayesian model averaging of Bayesian network classifiers over multiple node-orders: application to sparse datasets , 2005, IEEE Trans. Syst. Man Cybern. Part B.

[14]  Ercan Oztemel,et al.  Control chart pattern recognition using neural networks , 1992 .

[15]  Ramón López de Mántaras,et al.  TAN Classifiers Based on Decomposable Distributions , 2005, Machine Learning.

[16]  Henry Tirri,et al.  Comparing Soft Computing Methods in Prediction of Manufacturing Data , 1998, IEA/AIE.

[17]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[18]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[19]  Jordi Vitrià,et al.  Bayesian Classification for Inspection of Industrial Products , 2002, CCIA.

[20]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[21]  J. Kruskal.  On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .