A comparative study on swarm intelligence for structure learning of Bayesian networks

A Bayesian network (BN) is an important probabilistic model in the field of artificial intelligence and a powerful formalism used to describe uncertainty in the real world. As science and technology develop, considerable data on complex systems have been acquired by various means, which presents a significant challenge regarding how to accurately and robustly learn a network structure for a complex system. To address this challenge, many BN structure learning methods based on swarm intelligence have been developed. In this study, we perform a systematic comparison of three typical methods based on ant colony optimization, artificial bee colony algorithm, and bacterial foraging optimization. First, we analyze and summarize their main characteristics from the perspective of stochastic searching. Second, we conduct thorough experimental comparisons to examine the roles of different mechanisms in each method by means of multiaspect metrics, i.e., the K2 score, structural differences, and execution time. Next, we perform further experiments to validate the robustness of different algorithms on some benchmark data sets with noise. Finally, we present the prospects and references for researchers who are engaged in learning BN networks.

[1]  Stig K. Andersen,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[2]  Salma Jamoussi,et al.  Particle swarm optimization based method for Bayesian Network structure learning , 2013, 2013 5th International Conference on Modeling, Simulation and Applied Optimization (ICMSAO).

[3]  Thomas A. Runkler,et al.  Using a Local Discovery Ant Algorithm for Bayesian Network Structure Learning , 2009, IEEE Transactions on Evolutionary Computation.

[4]  Tong Wang,et al.  A heuristic method for learning Bayesian networks using discrete particle swarm optimization , 2010, Knowledge and Information Systems.

[5]  Jing Zhao,et al.  Structure Learning of Bayesian Networks Based on Discrete Binary Quantum-Behaved Particle Swarm Optimization Algorithm , 2009, 2009 Fifth International Conference on Natural Computation.

[6]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[7]  Baocai Yin,et al.  Structural learning of Bayesian networks by bacterial foraging optimization , 2016, Int. J. Approx. Reason..

[8]  Qiang Shen,et al.  Learning Bayesian Network Equivalence Classes with Ant Colony Optimization , 2009, J. Artif. Intell. Res..

[9]  Dervis Karaboga,et al.  AN IDEA BASED ON HONEY BEE SWARM FOR NUMERICAL OPTIMIZATION , 2005 .

[10]  Ji Jun A Bayesian Network Learning Algorithm Based on Independence Test and Ant Colony Optimization , 2009 .

[11]  Joseph Ramsey,et al.  Bayesian networks for fMRI: A primer , 2014, NeuroImage.

[12]  Kwong-Sak Leung,et al.  Using Evolutionary Programming and Minimum Description Length Principle for Data Mining of Bayesian Networks , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Qiang Shen,et al.  Learning Bayesian networks: approaches and issues , 2011, The Knowledge Engineering Review.

[14]  Zheng Qin,et al.  Learning Bayesian Network Structures with Discrete Particle Swarm Optimization Algorithm , 2007, 2007 IEEE Symposium on Foundations of Computational Intelligence.

[15]  Concha Bielza,et al.  Bayesian networks in neuroscience: a survey , 2014, Front. Comput. Neurosci..

[16]  Xiao-Lin Li A Particle Swarm Optimization and Immune Theory-Based Algorithm for Structure Learning of Bayesian Networks , 2010 .

[17]  Liu Chun-nian A Tabu-search Based Bayesian Network Structure Learning Algorithm , 2011 .

[18]  Concha Bielza,et al.  A review on evolutionary algorithms in Bayesian network learning and inference tasks , 2013, Inf. Sci..

[19]  Baocai Yin,et al.  Artificial Bee Colony Algorithm Merged with Pheromone Communication Mechanism for the 0-1 Multidimensional Knapsack Problem , 2013 .

[20]  J. Suzuki Learning Bayesian Belief Networks Based on the Minimum Description Length Principle: Basic Properties , 1999 .

[21]  Miguel A. Vega-Rodríguez,et al.  A Comparative Study on Multiobjective Swarm Intelligence for the Routing and Wavelength Assignment Problem , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[22]  R. W. Robinson Counting unlabeled acyclic digraphs , 1977 .

[23]  Jun-Zhong Ji,et al.  A Bayesian Network Learning Algorithm Based on Independence Test and Ant Colony Optimization: A Bayesian Network Learning Algorithm Based on Independence Test and Ant Colony Optimization , 2009 .

[24]  Fernando José Von Zuben,et al.  An immune-inspired approach to Bayesian networks , 2005, Fifth International Conference on Hybrid Intelligent Systems (HIS'05).

[25]  Jose Miguel Puerta,et al.  Ant colony optimization for learning Bayesian networks , 2002, Int. J. Approx. Reason..

[26]  Chunnian Liu,et al.  A hybrid method for learning Bayesian networks based on ant colony optimization , 2011, Appl. Soft Comput..

[27]  Laura E. Brown,et al.  Scaling-Up Bayesian Network Learning to Thousands of Variables Using Local Learning Techniques , 2003 .

[28]  Benoît Iung,et al.  Overview on Bayesian networks applications for dependability, risk analysis and maintenance areas , 2012, Eng. Appl. Artif. Intell..

[29]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[30]  Andrew J. Bulpitt,et al.  A Primer on Learning in Bayesian Networks for Computational Biology , 2007, PLoS Comput. Biol..

[31]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[32]  Fernando José Von Zuben,et al.  Copt-aiNet and the Gene Ordering Problem , 2003, WOB.

[33]  David J. Spiegelhalter,et al.  Bayesian analysis in expert systems , 1993 .

[34]  Shaogang Gong,et al.  Facial expression recognition based on Local Binary Patterns: A comprehensive study , 2009, Image Vis. Comput..

[35]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[36]  Sebastián Ventura,et al.  Educational Data Mining: A Review of the State of the Art , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[37]  Wray L. Buntine A Guide to the Literature on Learning Probabilistic Networks from Data , 1996, IEEE Trans. Knowl. Data Eng..

[38]  Weiru Liu,et al.  Learning belief networks from data: an information theory based approach , 1997, CIKM '97.

[39]  José A. Gámez,et al.  Learning Bayesian networks by Ant Colony Optimisation: searching in two different spaces , 2002 .

[40]  John A. W. McCall,et al.  Two novel Ant Colony Optimization approaches for Bayesian network structure learning , 2010, IEEE Congress on Evolutionary Computation.

[41]  Bart Baesens,et al.  Editorial survey: swarm intelligence for data mining , 2010, Machine Learning.

[42]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[43]  Luis M. de Campos,et al.  A new approach for learning belief networks using independence criteria , 2000, Int. J. Approx. Reason..

[44]  J. Alcobé Incremental Hill-Climbing Search Applied to Bayesian Network Structure Learning , 2004 .

[45]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[46]  Chunnian Liu,et al.  An artificial bee colony algorithm for learning Bayesian networks , 2012, Soft Computing.

[47]  Kazuyuki Mori,et al.  Immune Algorithm with Searching Diversity and its Application to Resource Allocation Problem , 1993 .

[48]  James Kennedy,et al.  Particle swarm optimization , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[49]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[50]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[51]  Gregory Gutin,et al.  Digraphs - theory, algorithms and applications , 2002 .

[52]  Kevin M. Passino,et al.  Biomimicry of bacterial foraging for distributed optimization and control , 2002 .