Using cooperative coevolution for data mining of Bayesian networks

Bayesian networks are formal knowledge representation tools that provide reasoning under uncertainty. The applications of Bayesian networks are widespread, including data mining, information retrieval, and various diagnostic systems. Although Bayesian networks are useful, the learning problem, namely to construct a network automatically from data, remains a difficult problem. Recently, some researchers have adopted evolutionary computation for learning. However, the drawback is that the approach is slow. In this chapter, we propose a hybrid framework for Bayesian network learning. By combining the merits of two different learning approaches, we expect an improvement in learning speed. In brief, the new learning algorithm consists of two phases: the conditional independence (CI) test phase and the search phase. In the CI test phase, we conduct dependency analysis, which helps to reduce the search space. In the search phase, we perform model searching using an evolutionary approach, called cooperative coevolution. When comparing our new algorithm with an existing algorithm, we find that our algorithm performs faster and is more accurate in many cases.

[1]  Eric Horvitz,et al.  Inferring Informational Goals from Free-Text Queries: A Bayesian Approach , 1998, UAI.

[2]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[3]  Zbigniew Michalewicz,et al.  Evolutionary Computation 1 , 2018 .

[4]  J. Suzuki Learning Bayesian Belief Networks Based on the Minimum Description Length Principle: Basic Properties , 1999 .

[5]  Judea Pearl,et al.  Chapter 2 – BAYESIAN INFERENCE , 1988 .

[6]  Weiru Liu,et al.  Learning belief networks from data: an information theory based approach , 1997, CIKM '97.

[7]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[8]  Stuart L. Crawford,et al.  Constructor: A System for the Induction of Probabilistic Models , 1990, AAAI.

[9]  Klaus-Uwe Höffgen,et al.  Learning and robust learning of product distributions , 1993, COLT '93.

[10]  David B. Fogel What is evolutionary computation , 1995 .

[11]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Jie Cheng,et al.  Learning Bayesian Networks from Data: An Efficient Approach Based on Information Theory , 1999 .

[13]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[14]  Mitchell A. Potter,et al.  EVOLVING NEURAL NETWORKS WITH COLLABORATIVE SPECIES , 2006 .

[15]  Moninder Singh,et al.  An Algorithm for the Construction of Bayesian Network Structures from Data , 1993, UAI.

[16]  Finn Verner Jensen,et al.  Introduction to Bayesian Networks , 2008, Innovations in Bayesian Networks.

[17]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[18]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[19]  Mitchell A. Potter,et al.  The design and analysis of a computational model of cooperative coevolution , 1997 .

[20]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[21]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 : Advanced Algorithms and Operators , 2000 .

[22]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[23]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[24]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 , 2000 .

[25]  John J. Grefenstette,et al.  A Coevolutionary Approach to Learning Sequential Decision Rules , 1995, ICGA.

[26]  David J. Spiegelhalter,et al.  Local computations with probabilities on graphical structures and their application to expert systems , 1990 .

[27]  Marek J. Druzdzel,et al.  A Hybrid Anytime Algorithm for the Construction of Causal Models From Sparse Data , 1999, UAI.

[28]  G. P. Beaumont,et al.  Statistical tests : an introduction with Minitab commentary , 1997 .

[29]  Kwong-Sak Leung,et al.  Using Evolutionary Programming and Minimum Description Length Principle for Data Mining of Bayesian Networks , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[31]  Jin Tian,et al.  A Branch-and-Bound Algorithm for MDL Learning Bayesian Networks , 2000, UAI.

[32]  A. Agresti,et al.  Categorical Data Analysis , 1991, International Encyclopedia of Statistical Science.

[33]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[34]  David Beasley,et al.  An overview of genetic algorithms: Part 1 , 1993 .

[35]  Chris Spatz,et al.  Basic Statistics: Tales of Distributions , 1981 .

[36]  Max Henrion,et al.  Propagating uncertainty in bayesian networks by probabilistic logic sampling , 1986, UAI.

[37]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[38]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[39]  Michael P. Wellman,et al.  Bayesian networks , 1995, CACM.