Ant colony optimization for learning Bayesian networks

One important approach to learning Bayesian networks (BNs) from data uses a scoring metric to evaluate the fitness of any given candidate network for the data base, and applies a search procedure to explore the set of candidate networks. The most usual search methods are greedy hill climbing, either deterministic or stochastic, although other techniques have also been used. In this paper we propose a new algorithm for learning BNs based on a recently introduced metaheuristic, which has been successfully applied to solve a variety of combinatorial optimization problems: ant colony optimization (ACO). We describe all the elements necessary to tackle our learning problem using this metaheuristic, and experimentally compare the performance of our ACO-based algorithm with other algorithms used in the literature. The experimental work is carried out using three different domains: ALARM, INSURANCE and BOBLO.

[1]  Pedro Larrañaga,et al.  Learning Bayesian networks in the space of structures by estimation of distribution algorithms , 2003, Int. J. Intell. Syst..

[2]  Nir Friedman,et al.  Being Bayesian about Network Structure , 2000, UAI.

[3]  Solomon Kullback,et al.  Information Theory and Statistics , 1970, The Mathematical Gazette.

[4]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[5]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[6]  Jose Miguel Puerta,et al.  Searching for the best elimination sequence in Bayesian networks by using ant colony optimization , 2002, Pattern Recognit. Lett..

[7]  Luis M. de Campos,et al.  A new approach for learning belief networks using independence criteria , 2000, Int. J. Approx. Reason..

[8]  Stuart J. Russell,et al.  Adaptive Probabilistic Networks with Hidden Variables , 1997, Machine Learning.

[9]  Judea Pearl,et al.  Equivalence and Synthesis of Causal Models , 1990, UAI.

[10]  Doug Fisher,et al.  Learning from Data: Artificial Intelligence and Statistics V , 1996 .

[11]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[12]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[13]  Wray L. Buntine.  A Guide to the Literature on Learning Probabilistic Networks from Data , 1996, IEEE Trans. Knowl. Data Eng..

[14]  Luis M. de Campos,et al.  Independency relationships and learning algorithms for singly connected networks , 1998, J. Exp. Theor. Artif. Intell..

[15]  Marco Dorigo,et al.  The ant colony optimization meta-heuristic , 1999 .

[16]  Luca Maria Gambardella,et al.  Ant Algorithms for Discrete Optimization , 1999, Artificial Life.

[17]  Riccardo Poli,et al.  New ideas in optimization , 1999 .

[18]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[19]  R. Bouckaert.  Bayesian belief networks : from construction to inference , 1995 .

[20]  Pedro Larrañaga,et al.  Learning Bayesian network structures by searching for the best ordering with genetic algorithms , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[21]  Moninder Singh,et al.  Construction of Bayesian network structures from data: A brief survey and an efficient algorithm , 1995, Int. J. Approx. Reason..

[22]  David Heckerman,et al.  Learning Bayesian Networks: Search Methods and Experimental Results , 1995 .

[23]  David Maxwell Chickering,et al.  Learning Bayesian Networks is NP-Complete , 2016, AISTATS.

[24]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[25]  Andrew P. Sage,et al.  Uncertainty in Artificial Intelligence , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[26]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[27]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[28]  Luis M. de Campos,et al.  A hybrid methodology for learning belief networks: BENEDICT , 2001, Int. J. Approx. Reason..

[29]  Marek J. Druzdzel,et al.  A Hybrid Anytime Algorithm for the Construction of Causal Models From Sparse Data , 1999, UAI.

[30]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[31]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[32]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[33]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[34]  Franz von Kutschera,et al.  Causation , 1993, J. Philos. Log..

[35]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Heinz Mühlenbein,et al.  The Equation for Response to Selection and Its Use for Prediction , 1997, Evolutionary Computation.

[37]  Luis M. de Campos,et al.  The Search of Causal Orderings: A Short Cut for Learning Belief Networks , 2001, ECSQARU.

[38]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[39]  Shumeet Baluja,et al.  A Method for Integrating Genetic Search Based Function Optimization and Competitive Learning , 1994 .