Evolving dynamic Bayesian networks with Multi-objective genetic algorithms

A dynamic Bayesian network (DBN) is a probabilistic network that models interdependent entities that change over time. Given example sequences of multivariate data, we use a genetic algorithm to synthesize a network structure that models the causal relationships that explain the sequence. We use a multi-objective evaluation strategy with a genetic algorithm. The multi-objective criteria are a network's probabilistic score and structural complexity score. Our use of Pareto ranking is ideal for this application, because it naturally balances the effect of the likelihood and structural simplicity terms used in the BIC network evaluation heuristic. We use a basic structural scoring formula, which tries to keep the number of links in the network approximately equivalent to the number of variables. We also use a simple representation that favors sparsely connected networks similar in structure to those modeling biological phenomenon. Our experiments show promising results when evolving networks ranging from 10 to 30 variables, using a maximal connectivity of between 3 and 4 parents per node. The results from the multi-objective GA were superior to those obtained with a single objective GA.

[1]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[2]  Kevin P. Murphy,et al.  Dynamic Bayesian Networks for Audio-Visual Speech Recognition , 2002, EURASIP J. Adv. Signal Process..

[3]  Xiaohui Liu,et al.  Learning Dynamic Bayesian Networks from Multivariate Time Series with Changing Dependencies , 2003, IDA.

[4]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[5]  Carlos Cotta,et al.  Towards a More Efficient Evolutionary Induction of Bayesian Networks , 2002, PPSN.

[6]  Xiaohui Liu,et al.  Evolutionary learning of dynamic probabilistic models with large time lags , 2001, Int. J. Intell. Syst..

[7]  Andrew W. Moore,et al.  Bayesian networks for lossless dataset compression , 1999, KDD '99.

[8]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[9]  Hsuan-Cheng Huang,et al.  GeneNetwork: an interactive tool for reconstruction of genetic networks using microarray data , 2004, Bioinform..

[10]  Xiaohui Liu,et al.  Spatial Operators for Evolving Dynamic Bayesian Networks from Spatio-temporal Data , 2003, GECCO.

[11]  R. Bouckaert.  Minimum Description Length Principle , 1994 .

[12]  Todd Andrew Stephenson,et al.  An Introduction to Bayesian Network Theory and Usage , 2000 .

[13]  David Maxwell Chickering,et al.  Learning Bayesian Networks is NP-Complete , 2016, AISTATS.

[14]  Eugene Charniak,et al.  Bayesian Networks without Tears , 1991, AI Mag..

[15]  Dirk Husmeier,et al.  Sensitivity and specificity of inferring genetic regulatory interactions from microarray experiments with dynamic Bayesian networks , 2003, Bioinform..

[16]  William H. Hsu,et al.  A Survey of Algorithms for Real-Time Bayesian Network Inference , 2002 .

[17]  William H. Hsu,et al.  A Permutation Genetic Algorithm For Variable Ordering In Learning Bayesian Networks From Data , 2002, GECCO.

[18]  J. Rissanen.  Stochastic Complexity and Modeling , 1986 .

[19]  Paul P. Wang,et al.  Advances to Bayesian network inference for generating causal networks from observational biological data , 2004, Bioinform..

[20]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[21]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[22]  Kevin P. Murphy,et al.  Learning the Structure of Dynamic Probabilistic Networks , 1998, UAI.

[23]  Kevin Murphy,et al.  Bayes net toolbox for Matlab , 1999 .

[24]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[25]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[26]  Wray L. Buntine.  Operations for Learning with Graphical Models , 1994, J. Artif. Intell. Res..

[27]  Gary B. Lamont,et al.  Multiobjective Evolutionary Algorithms: Analyzing the State-of-the-Art , 2000, Evolutionary Computation.

[28]  Pedro Larrañaga,et al.  Predicting survival in malignant skin melanoma using Bayesian networks automatically induced by genetic algorithms. An empirical comparison between different approaches , 1998, Artif. Intell. Medicine.

[29]  Kevin Murphy,et al.  Dynamic Bayesian Networks , 2002 .

[30]  Michael C. Horsch,et al.  Dynamic Bayesian networks , 1990 .

[31]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[32]  Richard Scheines,et al.  Genetic Algorithm Search Over Causal Models , 2002 .

[33]  Po Shun Ngan,et al.  Discover Probabilistic Knowledge from Databases Using Evolutionary Computation and Minimum Description Length Principle , 2006 .

[34]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[35]  Satoru Miyano,et al.  Inferring gene networks from time series microarray data using dynamic Bayesian networks , 2003, Briefings Bioinform..

[36]  Peter J. Fleming,et al.  An Overview of Evolutionary Algorithms in Multiobjective Optimization , 1995, Evolutionary Computation.

[37]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Illtyd Trethowan.  Causality , 1938 .

[39]  Aurélien Mazurie,et al.  Gene networks inference using dynamic Bayesian networks , 2003, ECCB.

[40]  Dirk Thierens,et al.  Building a GA from Design Principles for Learning Bayesian Networks , 2003, GECCO.

[41]  R. Albert,et al.  The large-scale organization of metabolic networks , 2000, Nature.

[42]  Lonnie Chrisman,et al.  A Roadmap to Research on Bayesian Networks and other Decomposable Probabilistic Models , 1998 .

[43]  Kathryn B. Laskey,et al.  Learning Bayesian networks from incomplete data using evolutionary algorithms , 1999 .

[44]  Frédéric Gruau,et al.  Genetic micro programming of neural networks , 1994 .

[45]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1998, Learning in Graphical Models.

[46]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[47]  Gary B. Lamont,et al.  Evolutionary Algorithms for Solving Multi-Objective Problems , 2002, Genetic Algorithms and Evolutionary Computation.

[48]  Kwong-Sak Leung,et al.  A Hybrid Data Mining Approach To Discover Bayesian Networks Using Evolutionary Programming , 2002, GECCO.