Improved Bayesian Network Structure Learning with Node Ordering via K2 Algorithm

The precise construction of Bayesian network classifier from database is an NP-hard problem and still one of the most exciting challenges. K2 algorithm can reduce search space effectively, improve learning efficiency, but it requires the initial node ordering as input, which is very limited by the absence of the priori information. On the other hand, search process of K2 algorithm uses a greedy search strategy and solutions are easy to fall into local optimization. In this paper, we present an improved Bayesian network structure learning with node ordering via K2 algorithm. This algorithm generates an effective node ordering as input based on conditional mutual information. The K2 algorithm is also improved combining with Simulated Annealing algorithm in order to avoid falling into the local optimization. Experimental results over two benchmark networks Asia and Alarm show that this new improved algorithm has higher classification accuracy and better degree of data matching.

[1]  Geoffrey I. Webb,et al.  Encyclopedia of Machine Learning , 2011, Encyclopedia of Machine Learning.

[2]  David Maxwell Chickering,et al.  Optimal Structure Identification With Greedy Search , 2002, J. Mach. Learn. Res..

[3]  Kwong-Sak Leung,et al.  An efficient data mining method for learning Bayesian networks using an evolutionary algorithm-based hybrid approach , 2004, IEEE Transactions on Evolutionary Computation.

[4]  Adnan Darwiche,et al.  A differential approach to inference in Bayesian networks , 2000, JACM.

[5]  David Heckerman,et al.  Learning Bayesian Networks: Search Methods and Experimental Results , 1995 .

[6]  Wai Lam,et al.  Using Causal Information and Local Measures to Learn Bayesian Networks , 1993, UAI.

[7]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[8]  Xue-wen Chen,et al.  Improving Bayesian Network Structure Learning with Mutual Information-Based Node Ordering in the K2 Algorithm , 2008, IEEE Transactions on Knowledge and Data Engineering.

[9]  Ying Tan,et al.  Advances in Swarm Intelligence , 2016, Lecture Notes in Computer Science.

[10]  Harald Steck,et al.  Learning the Bayesian Network Structure: Dirichlet Prior versus Data , 2008, UAI 2008.

[11]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[12]  Jiawei Han,et al.  SRDA: An Efficient Algorithm for Large-Scale Discriminant Analysis , 2008, IEEE Transactions on Knowledge and Data Engineering.

[13]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..

[14]  Boaz Lerner,et al.  INVESTIGATION OF THE K2 ALGORITHM IN LEARNING BAYESIAN NETWORK CLASSIFIERS , 2011, Appl. Artif. Intell..

[15]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[16]  Kathryn A. Dowsland,et al.  Simulated Annealing , 1989, Encyclopedia of GIS.

[17]  Hua Yu,et al.  Poison Identification Based on Bayesian Network: A Novel Improvement on K2 Algorithm via Markov Blanket , 2013, ICSI.

[18]  Jose Miguel Puerta,et al.  Ant colony optimization for learning Bayesian networks , 2002, Int. J. Approx. Reason..

[19]  Jorma Rissanen,et al.  Minimum Description Length Principle , 2010, Encyclopedia of Machine Learning.

[20]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[21]  Zheng Yun,et al.  Improved MDL Score for Learning of Bayesian Networks , 2004 .