Bayesian network structure learning using cooperative coevolution

We propose a cooperative-coevolution - Parisian trend - algorithm, IMPEA (Independence Model based Parisian EA), to the problem of Bayesian networks structure estimation. It is based on an intermediate stage which consists of evaluating an independence model of the data to be modelled. The Parisian cooperative coevolution is particularly well suited to the structure of this intermediate problem, and allows to represent an independence model with help of a whole population, each individual being an independence statement, i.e. a component of the independence model. Once an independence model is estimated, a Bayesian network can be built. This two level resolution of the complex problem of Bayesian network structure estimation has the major advantage to avoid the difficult problem of direct acyclic graph representation within an evolutionary algorithm, which causes many troubles related to constraints handling and slows down algorithms. Comparative results with a deterministic algorithm, PC, on two test cases (including the Insurance BN benchmark), prove the efficiency of IMPEA, which provides better results than PC in a comparable computation time, and which is able to tackle more complex issues than PC.

[1]  P. Spirtes,et al.  Causation, Prediction, and Search, 2nd Edition , 2001 .

[2]  David Maxwell Chickering,et al.  Large-Sample Learning of Bayesian Networks is NP-Hard , 2002, J. Mach. Learn. Res..

[3]  Peng Yu,et al.  Learning dynamic Bayesian network with immune evolutionary algorithm , 2005, 2005 International Conference on Machine Learning and Cybernetics.

[4]  Gregory F. Cooper,et al.  A Bayesian Method for the Induction of Probabilistic Networks from Data , 1992 .

[5]  Judea Pearl,et al.  A Theory of Inferred Causation , 1991, KR.

[6]  Michal Linial,et al.  Using Bayesian Networks to Analyze Expression Data , 2000, J. Comput. Biol..

[7]  Philippe Leray,et al.  Étude Comparative d’Algorithmes d’Apprentissage de Structure dans les Réseaux Bayésiens , 2004 .

[8]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[9]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[10]  David E. Goldberg,et al.  Genetic Algorithms with Sharing for Multimodalfunction Optimization , 1987, ICGA.

[11]  Stuart J. Russell,et al.  Adaptive Probabilistic Networks with Hidden Variables , 1997, Machine Learning.

[12]  Cédric Baudrit,et al.  A Dynamic Bayesian Network to Represent a Ripening Process of a Soft Mould Cheese , 2008, KES.

[13]  Allan Tucker,et al.  Extending Evolutionary Programming methods to the learning of dynamic Bayesian networks , 1999 .

[14]  Kwong-Sak Leung,et al.  An efficient data mining method for learning Bayesian networks using an evolutionary algorithm-based hybrid approach , 2004, IEEE Transactions on Evolutionary Computation.

[15]  Marie-Jeanne Lesot,et al.  Dynamic flies: a new pattern recognition tool applied to stereo sequence processing , 2002, Pattern Recognit. Lett..

[16]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[17]  Kathryn B. Laskey,et al.  Learning Bayesian networks from incomplete data using evolutionary algorithms , 1999 .

[18]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Kalyanmoy Deb,et al.  An Investigation of Niche and Species Formation in Genetic Function Optimization , 1989, ICGA.

[20]  Evelyne Lutton,et al.  A Genetic Algorithm with Sharing for the Detection of 2D Geometric Primitives in Images , 1995, Artificial Evolution.

[21]  R. W. Robinson.  Counting unlabeled acyclic digraphs , 1977 .

[22]  Brian J. Ross,et al.  Evolving dynamic Bayesian networks with Multi-objective genetic algorithms , 2007, Applied Intelligence.

[23]  Weiru Liu,et al.  Learning belief networks from data: an information theory based approach , 1997, CIKM '97.

[24]  Sun-Chong Wang,et al.  Learning Bayesian Networks by Lamarckian Genetic Algorithm and Its Application to Yeast Cell-Cycle Gene Network Reconstruction from Time-Series Microarray Data , 2004, BioADIT.

[25]  Edmund K. Burke,et al.  The Cooperative Royal Road: Avoiding Hitchhiking , 2007, Artificial Evolution.

[26]  Kevin Murphy,et al.  Bayes net toolbox for Matlab , 1999 .

[27]  Nir Friedman,et al.  Learning Belief Networks in the Presence of Missing Values and Hidden Variables , 1997, ICML.