Learning Bayesian networks from incomplete data using evolutionary algorithms

This paper describes an evolutionary algorithm approach to learning Bayesian networks from incomplete data. This problem is characterized by a huge solution space with a highly multimodal landscape. State-of-the-art approaches all involve using deterministic approaches such as the expectation-maximization algorithm. These approaches are guaranteed to find local maxima, but do not explore the landscape for other modes. Our approach evolves the structure of the network and the missing data. We use a factorial design to choose a good set of parameters for selection, crossover, and mutation. We show that our algorithm produces accurate results for a classification problem with missing data.

[1]  H. Hartley.  Maximum Likelihood Estimation from Incomplete Data , 1958 .

[2]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[3]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[4]  D. Rubin,et al.  Statistical Analysis with Missing Data , 1988 .

[5]  Roderick J. A. Little,et al.  Statistical Analysis with Missing Data , 1988 .

[6]  Gilbert Syswerda,et al.  Uniform Crossover in Genetic Algorithms , 1989, ICGA.

[7]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[8]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[9]  T. Fearn,et al.  Bayesian statistics : principles, models, and applications , 1990 .

[10]  Kenneth A. De Jong,et al.  An Analysis of the Interacting Roles of Population Size and Crossover in Genetic Algorithms , 1990, PPSN.

[11]  F. Guess.  Bayesian Statistics: Principles, Models, and Applications , 1990 .

[12]  Kalyanmoy Deb,et al.  A Comparative Analysis of Selection Schemes Used in Genetic Algorithms , 1990, FOGA.

[13]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[14]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms , 1994 .

[15]  S. Lauritzen.  The EM algorithm for graphical association models with missing data , 1995 .

[16]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[17]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Hybrid Genetic Algorithms , 1995, AISTATS.

[18]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Nir Friedman,et al.  Learning Belief Networks in the Presence of Missing Values and Hidden Variables , 1997, ICML.

[20]  Jim Q. Smith,et al.  On the Geometry of Bayesian Graphical Models with Hidden Variables , 1998, UAI.

[21]  Nir Friedman,et al.  The Bayesian Structural EM Algorithm , 1998, UAI.

[22]  Dan Geiger,et al.  Graphical Models and Exponential Families , 1998, UAI.

[23]  William M. Spears,et al.  Simple Subpopulation Schemes , 1998 .