Learning bayesian network structures with GOMEA

Bayesian networks (BNs) are probabilistic graphical models which are widely used for knowledge representation and decision making tasks, especially in the presence of uncertainty. Finding or learning the structure of BNs from data is an NP-hard problem. Evolutionary algorithms (EAs) have been extensively used to automate the learning process. In this paper, we consider the use of the Gene-Pool Optimal Mixing Evolutionary Algorithm (GOMEA). GOMEA is a relatively new type of EA that belongs to the class of model-based EAs. The model used in GOMEA is aimed at modeling the dependency structure between problem variables, so as to improve the efficiency and effectiveness of variation. This paper shows that the excellent performance of GOMEA transfers from well-known academic benchmark problems to the specific case of learning BNs from data due to its model-building capacities and the potential to compute partial evaluations when learning BNs. On commonly-used datasets of varying size, we find that GOMEA outperforms standard algorithms such as Order-based search (OBS), as well as other EAs, such as Genetic Algorithms (GAs) and Estimation of Distribution algorithms (EDAs), even when efficient local search techniques are added.

[1]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[2]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[3]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Hybrid Genetic Algorithms , 1995, AISTATS.

[4]  Pedro Larrañaga,et al.  Decomposing Bayesian networks: triangulation of the moral graph with genetic algorithms , 1997, Stat. Comput..

[5]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[6]  J. Rissanen,et al.  Modeling By Shortest Data Description* , 1978, Autom..

[7]  Dirk Thierens,et al.  Building a GA from Design Principles for Learning Bayesian Networks , 2003, GECCO.

[8]  Constantin F. Aliferis,et al.  The max-min hill-climbing Bayesian network structure learning algorithm , 2006, Machine Learning.

[9]  R. Bouckaert.  Bayesian belief networks : from construction to inference , 1995 .

[10]  H. Akaike.  A new look at the statistical model identification , 1974 .

[11]  P. Spirtes,et al.  An Algorithm for Fast Recovery of Sparse Causal Graphs , 1991 .

[12]  Marin Golub,et al.  Evaluation of Crossover Operator Performance in Genetic Algorithms with Binary Representation , 2011, ICIC.

[13]  Dirk Thierens,et al.  More concise and robust linkage learning by filtering and combining linkage hierarchies , 2013, GECCO '13.

[14]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Dirk Thierens,et al.  A Skeleton-Based Approach to Learning Bayesian Networks from Data , 2003, PKDD.

[16]  Kumara Sastry,et al.  Linkage Learning via Probabilistic Modeling in the Extended Compact Genetic Algorithm (ECGA) , 2006, Scalable Optimization via Probabilistic Modeling.

[17]  Wray L. Buntine.  Operations for Learning with Graphical Models , 1994, J. Artif. Intell. Res..

[18]  Judea Pearl,et al.  Chapter 2 – BAYESIAN INFERENCE , 1988 .

[19]  Daphne Koller,et al.  Ordering-Based Search: A Simple and Effective Algorithm for Learning Bayesian Networks , 2005, UAI.

[20]  Peter A. N. Bosman,et al.  Medium-Voltage Distribution Network Expansion Planning with Gene-pool Optimal Mixing Evolutionary Algorithms , 2013, Artificial Evolution.

[21]  Peter C. Nelson,et al.  The role of crossover operator in bayesian network structure learning performance: a comprehensive comparative study and new insights , 2017, GECCO.

[22]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[23]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[24]  Dirk Thierens,et al.  Optimal mixing evolutionary algorithms , 2011, GECCO '11.

[25]  Nir Friedman,et al.  Learning Bayesian Network Structure from Massive Datasets: The "Sparse Candidate" Algorithm , 1999, UAI.

[26]  Peter A. N. Bosman,et al.  Exploiting Linkage Information and Problem-Specific Knowledge in Evolutionary Distribution Network Expansion Planning , 2018, Evolutionary Computation.