The Impact of Exact Probabilistic Learning Algorithms in EDAs Based on Bayesian Networks

This paper discusses exact learning of Bayesian networks in estimation of distribution algorithms. The estimation of Bayesian network algorithm (EBNA) is used to analyze the impact of learning the optimal (exact) structure in the search. By applying recently introduced methods that allow learning optimal Bayesian networks, we investigate two important issues in EDAs. First, we analyze the question of whether learning more accurate (exact) models of the dependencies implies a better performance of EDAs. Secondly, we are able to study the way in which the problem structure is translated into the probabilistic model when exact learning is accomplished. The results obtained reveal that the quality of the problem information captured by the probability model can improve when the accuracy of the learning algorithm employed is increased. However, improvements in model accuracy do not always imply a more efficient search.

[1]  Pedro Larrañaga,et al.  Feature Weighting for Nearest Neighbor by Estimation of Distribution Algorithms , 2002, Estimation of Distribution Algorithms.

[2]  Alberto Ochoa,et al.  Linking Entropy to Estimation of Distribution Algorithms , 2006, Towards a New Evolutionary Computation.

[3]  Pedro Larrañaga,et al.  Exact Bayesian network learning in estimation of distribution algorithms , 2007, 2007 IEEE Congress on Evolutionary Computation.

[4]  Pedro Larrañaga,et al.  Component weighting functions for adaptive search with EDAs , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[5]  Pedro Larrañaga,et al.  Interactions and dependencies in estimation of distribution algorithms , 2005, 2005 IEEE Congress on Evolutionary Computation.

[6]  Pedro Larrañaga,et al.  Estimation of Distribution Algorithms , 2002, Genetic Algorithms and Evolutionary Computation.

[7]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[8]  D. Goldberg,et al.  BOA: the Bayesian optimization algorithm , 1999 .

[9]  Pedro Larrañaga,et al.  Estimation of distribution algorithms with affinity propagation methods , 2008 .

[10]  H. Mühlenbein,et al.  From Recombination of Genes to the Estimation of Distributions I. Binary Parameters , 1996, PPSN.

[11]  S. Baluja,et al.  Using Optimal Dependency-Trees for Combinatorial Optimization: Learning the Structure of the Search Space , 1997 .

[12]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[13]  Pedro Larrañaga,et al.  Protein Folding in 2-Dimensional Lattices with Estimation of Distribution Algorithms , 2004, ISBMDA.

[14]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[15]  David Starer,et al.  Artificial Neural Nets , 1995 .

[16]  Pedro Larrañaga,et al.  The Role of a Priori Information in the Minimization of Contact Potentials by Means of Estimation of Distribution Algorithms , 2007, EvoBIO.

[17]  Heinz Mühlenbein,et al.  The Estimation of Distributions and the Minimum Relative Entropy Principle , 2005, Evol. Comput..

[18]  Martin Pelikan,et al.  Hierarchical Bayesian optimization algorithm: toward a new generation of evolutionary algorithms , 2010, SICE 2003 Annual Conference (IEEE Cat. No.03TH8734).

[19]  Thilo Mahnig,et al.  Evolutionary Synthesis of Bayesian Networks for Optimization , 1999 .

[20]  Roberto Santana,et al.  Estimation of Distribution Algorithms with Kikuchi Approximations , 2005, Evolutionary Computation.

[21]  Andrew W. Moore,et al.  Finding optimal Bayesian networks by dynamic programming , 2005 .

[22]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[23]  Carlos Cotta,et al.  Protein Structure Prediction Using Evolutionary Algorithms Hybridized with Backtracking , 2009, IWANN.

[24]  Robert Elliott Smith,et al.  The limitations of distribution sampling for linkage learning , 2007, 2007 IEEE Congress on Evolutionary Computation.

[25]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[26]  Pedro Larrañaga,et al.  Protein Folding in Simplified Models With Estimation of Distribution Algorithms , 2008, IEEE Transactions on Evolutionary Computation.

[27]  Chung-Yao Chuang,et al.  On the effectiveness of distributions estimated by probabilistic model building , 2008, GECCO '08.

[28]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[29]  David E. Goldberg,et al.  Influence of selection and replacement strategies on linkage learning in BOA , 2007, 2007 IEEE Congress on Evolutionary Computation.

[30]  Jose A. Lozano,et al.  An Empirical Comparison of Discrete Estimation of Distribution Algorithms , 2002, Estimation of Distribution Algorithms.

[31]  Martin Pelikan,et al.  Analyzing probabilistic models in hierarchical BOA on traps and spin glasses , 2007, GECCO '07.

[32]  Dieter Fensel,et al.  Problem-Solving Methods , 2001, Lecture Notes in Computer Science.

[33]  Vincenzo Cutello,et al.  An Immune Algorithm for Protein Structure Prediction on Lattice Models , 2007, IEEE Transactions on Evolutionary Computation.

[34]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[35]  Alexander Mendiburu,et al.  Parallel implementation of EDAs based on probabilistic graphical models , 2005, IEEE Transactions on Evolutionary Computation.

[36]  Victor Maojo,et al.  Biological and Medical Data Analysis, 6th International Symposium, ISBMDA 2005, Aveiro, Portugal, November 10-11, 2005, Proceedings , 2005, ISBMDA.

[37]  A. Ochoa,et al.  A factorized distribution algorithm based on polytrees , 2000, Proceedings of the 2000 Congress on Evolutionary Computation. CEC00 (Cat. No.00TH8512).

[38]  Mikko Koivisto,et al.  Exact Bayesian Structure Discovery in Bayesian Networks , 2004, J. Mach. Learn. Res..

[39]  Hans-Paul Schwefel,et al.  Parallel Problem Solving from Nature — PPSN IV , 1996, Lecture Notes in Computer Science.

[40]  Kevin P. Murphy,et al.  Exact Bayesian structure learning from uncertain interventions , 2007, AISTATS.

[41]  K. Dill.  Theory for the folding and stability of globular proteins. , 1985, Biochemistry.

[42]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[43]  David E. Goldberg,et al.  Using Previous Models to Bias Structural Learning in the Hierarchical BOA , 2008, Evolutionary Computation.

[44]  Tomi Silander,et al.  A Simple Approach for Finding the Globally Optimal Bayesian Network Structure , 2006, UAI.

[45]  Heinz Mühlenbein,et al.  Schemata, Distributions and Graphical Models in Evolutionary Optimization , 1999, J. Heuristics.

[46]  David E. Goldberg,et al.  Hierarchical Bayesian Optimization Algorithm , 2006, Scalable Optimization via Probabilistic Modeling.