Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters

We present a new approach to structure learning in the field of Bayesian networks. We tackle the problem of the search for the best Bayesian network structure, given a database of cases, using the genetic algorithm philosophy for searching among alternative structures. We start by assuming an ordering between the nodes of the network structures. This assumption is necessary to guarantee that the networks that are created by the genetic algorithms are legal Bayesian network structures. Next, we release the ordering assumption by using a "repair operator" which converts illegal structures into legal ones. We present empirical results and analyze them statistically. The best results are obtained with an elitist genetic algorithm that contains a local optimizer.

[1]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms , 1994 .

[2]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[3]  Günter Rudolph,et al.  Convergence analysis of canonical genetic algorithms , 1994, IEEE Trans. Neural Networks.

[4]  Remco R. Bouckaert,et al.  Optimizing Causal Orderings for Generating DAGs from Data , 1992, UAI.

[5]  Bruce Abramson,et al.  The Topological Fusion of Bayes Nets , 1992, UAI.

[6]  Remco R. Bouckaert,et al.  Properties of Bayesian Belief Network Learning Algorithms , 1994, UAI.

[7]  R. W. Robinson.  Counting unlabeled acyclic digraphs , 1977 .

[8]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[9]  Finn Verner Jensen,et al.  Introduction to Bayesian Networks , 2008, Innovations in Bayesian Networks.

[10]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[11]  Luis M. de Campos,et al.  Learning with CASTLE , 1991, ECSQARU.

[12]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[13]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[14]  A. E. Eiben,et al.  Global Convergence of Genetic Algorithms: A Markov Chain Analysis , 1990, PPSN.

[15]  Stuart L. Crawford,et al.  Constructor: A System for the Induction of Probabilistic Models , 1990, AAAI.

[16]  Klaus-Uwe Höffgen,et al.  Learning and robust learning of product distributions , 1993, COLT '93.

[17]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[18]  Gregory Provan,et al.  Model Selection for Diagnosis and Treatment Using Temporal Influence Diagrams , 1994 .

[19]  R. Hartl.  A Global Convergence Proof for a Class of Genetic Algorithms , 1990 .

[20]  Gregory F. Cooper,et al.  An Entropy-driven System for Construction of Probabilistic Expert Systems from Databases , 1990, UAI.

[21]  Dag Wedelin.  Efficient Algorithms for Probabilistic Inference, Combinatorial Optimization and the Discovery of Causal Structure from Data , 1993 .

[22]  Constantin F. Aliferis,et al.  An Evaluation of an Algorithm for Inductive Learning of Bayesian Belief Networks Using Simulated Data Sets , 1994, UAI.

[23]  Marco Valtorta,et al.  A Parallel Constructor of Markov Networks , 1994 .

[24]  Gregory M. Provan,et al.  Learning Bayesian Networks Using Feature Selection , 1995, AISTATS.

[25]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[26]  Moninder Singh,et al.  Construction of Bayesian network structures from data: A brief survey and an efficient algorithm , 1995, Int. J. Approx. Reason..

[27]  David Heckerman,et al.  Learning Bayesian Networks: Search Methods and Experimental Results , 1995 .

[28]  Pedro Larrañaga,et al.  Structure learning approaches in Causal Probalistics Networks , 1993, ECSQARU.

[29]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine-mediated learning.

[30]  Joe Suzuki,et al.  A Construction of Bayesian Networks from Databases Based on an MDL Principle , 1993, UAI.

[31]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[32]  H. P. Schwefel,et al.  Numerische Optimierung von Computermodellen mittels der Evo-lutionsstrategie , 1977 .

[33]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[34]  Edward H. Herskovits,et al.  Computer-based probabilistic-network construction , 1992 .

[35]  Uffe Kjærulff,et al.  A Computational Scheme for Reasoning in Dynamic Probabilistic Networks , 1992, UAI.

[36]  Uday Kumar Chakraborty,et al.  Using Reliability Analysis to Estimate the Number of Generations to Convergence in Genetic Algorithms , 1993, Inf. Process. Lett..

[37]  Richard E. Neapolitan,et al.  Probabilistic reasoning in expert systems - theory and algorithms , 2012 .

[38]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[39]  Remco R. Bouckaert,et al.  Probalistic Network Construction Using the Minimum Description Length Principle , 1993, ECSQARU.

[40]  Russell G. Almond,et al.  Strategies for Graphical Model Selection , 1994 .

[41]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Hybrid Genetic Algorithms , 1995, AISTATS.

[42]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[43]  Wai Lam,et al.  Using Causal Information and Local Measures to Learn Bayesian Networks , 1993, UAI.

[44]  Hans-Paul Schwefel,et al.  Numerical Optimization of Computer Models , 1982 .

[45]  Moninder Singh,et al.  An Algorithm for the Construction of Bayesian Network Structures from Data , 1993, UAI.