What makes a problem hard for a genetic algorithm? Some anomalous results and their explanation

What makes a problem easy or hard for a genetic algorithm (GA)? This question has become increasingly important as people have tried to apply the GA to ever more diverse types of problems. Much previous work on this question has studied the relationship between GA performance and the structure of a given fitness function when it is expressed as aWalsh polynomial. The work of Bethke, Goldberg, and others has produced certain theoretical results about this relationship. In this article we review these theoretical results, and then discuss a number of seemingly anomalous experimental results reported by Tanese concerning the performance of the GA on a subclass of Walsh polynomials, some members of which were expected to be easy for the GA to optimize. Tanese found that the GA was poor at optimizing all functions in this subclass, that a partitioning of a single large population into a number of smaller independent populations seemed to improve performance, and that hillelimbing outperformed both the original and partitioned forms of the GA on these functions. These results seemed to contradict several commonly held expectations about GAs.We begin by reviewingschema processing in GAs. We then given an informal description of how Walsh analysis and Bethke's Walsh-schema transform relate to GA performance, and we discuss the relevance of this analysis for GA applications in optimization and machine learning. We then describe Tanese's surprising results, examine them experimentally and theoretically, and propose and evaluate some explanations. These explanations lead to a more fundamental question about GAs: what are the features of problems that determine the likelihood of successful GA performance?

[1]  J. Walsh.  A Closed Set of Normal Orthogonal Functions , 1923 .

[2]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[3]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[4]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[5]  Albert Donally Bethke,et al.  Genetic Algorithms as Function Optimizers , 1980 .

[6]  Stephen F. Smith,et al.  A learning system based on genetic adaptive algorithms , 1980 .

[7]  John J. Grefenstette,et al.  Genetic algorithms and their applications , 1987 .

[8]  R. Axelrod.  An Evolutionary Approach to Norms , 1986, American Political Science Review.

[9]  John H. Holland,et al.  Escaping brittleness: the possibilities of general-purpose learning algorithms applied to parallel rule-based systems , 1995 .

[10]  D. E. Goldberg,et al.  Simple Genetic Algorithms and the Minimal, Deceptive Problem , 1987 .

[11]  Lawrence Davis,et al.  Genetic Algorithms and Simulated Annealing , 1987 .

[12]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[13]  Y. C. Lee,et al.  Evolution, Learning And Cognition , 1988 .

[14]  Rajarshi Das,et al.  A Study of Control Parameters Affecting Online Performance of Genetic Algorithms for Function Optimization , 1989, ICGA.

[15]  J. David Schaffer,et al.  Proceedings of the third international conference on Genetic algorithms , 1989 .

[16]  David E. Goldberg,et al.  Sizing Populations for Serial and Parallel Genetic Algorithms , 1989, ICGA.

[17]  Daniel L. Stein,et al.  Lectures In The Sciences Of Complexity , 1989 .

[18]  John J. Grefenstette,et al.  How Genetic Algorithms Work: A Critical Look at Implicit Parallelism , 1989, ICGA.

[19]  David E. Goldberg,et al.  Genetic Algorithms and Walsh Functions: Part I, A Gentle Introduction , 1989, Complex Syst..

[20]  David E. Goldberg,et al.  Genetic Algorithms and Walsh Functions: Part II, Deception and Its Analysis , 1989, Complex Syst..

[21]  Reiko Tanese,et al.  Distributed Genetic Algorithms , 1989, ICGA.

[22]  John H. Holland,et al.  The Dynamics of Searches Directed by Genetic Algorithms , 1989 .

[23]  John H. Holland,et al.  Distributed genetic algorithms for function optimization , 1989 .

[24]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[25]  Peter M. Todd,et al.  Designing Neural Networks using Genetic Algorithms , 1989, ICGA.

[26]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[27]  Alan S. Perelson,et al.  Genetic Algorithms and the Immune System , 1990, PPSN.

[28]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[29]  M. Feldman,et al.  More on selection for and against recombination. , 1990, Theoretical population biology.

[30]  John R. Koza,et al.  Genetic programming: a paradigm for genetically breeding populations of computer programs to solve problems , 1990 .

[31]  Norman H. Packard,et al.  A Genetic Learning Algorithm for the Analysis of Complex Data , 1990, Complex Syst..

[32]  GUNAR E. LIEPINS,et al.  Representational issues in genetic optimization , 1990, J. Exp. Theor. Artif. Intell..

[33]  W. Daniel Hillis,et al.  Co-evolving parasites improve simulated evolution as an optimization procedure , 1990 .

[34]  L. Darrell Whitley,et al.  Fundamental Principles of Deception in Genetic Search , 1990, FOGA.

[35]  L. Darrell Whitley,et al.  Genetic Reinforcement Learning with Multilayer Neural Networks , 1991, ICGA.

[36]  Stewart W. Wilson.  GA-Easy Does Not Imply Steepest-Ascent Optimizable , 1991, ICGA.

[37]  Lashon B. Booker,et al.  Proceedings of the fourth international conference on Genetic algorithms , 1991 .

[38]  Kenneth A. De Jong,et al.  Learning Concept Classification Rules Using Genetic Algorithms , 1991, IJCAI.

[39]  Gunar E. Liepins,et al.  Polynomials, Basis Sets, and Deceptiveness in Genetic Algorithms , 1991, Complex Syst..

[40]  L. Darrell Whitley,et al.  The Only Challenging Problems Are Deceptive: Global Search by Solving Order-1 Hyperplanes , 1991, ICGA.

[41]  Melanie Mitchell,et al.  The Performance of Genetic Algorithms on Walsh Polynomials: Some Anomalous Results and Their Explanation , 1991, ICGA.

[42]  Melanie Mitchell,et al.  The royal road for genetic algorithms: Fitness landscapes and GA performance , 1991 .