Evolutionary algorithm characterization in real parameter optimization problems

This paper deals with the constant problem of establishing a usable and reliable evolutionary algorithm (EA) characterization procedure so that final users like engineers, mathematicians or physicists can have more specific information to choose the most suitable EA for a given problem. The practical goal behind this work is to provide insights into relevant features of fitness landscapes and their relationship to the performance of different algorithms. This should help users to minimize the typical initial stage in which they apply a well-known EA, or a modified version of it, to the functions they want to optimize without really taking into account its suitability to the particular features of the problem. This trial and error procedure is usually due to a lack of objective and detailed characterizations of the algorithms in the literature in terms of the types of functions or landscape characteristics they are well suited to handle and, more importantly, the types for which they are not appropriate. Specifically, the influence of separability and modality of the fitness landscapes on the behaviour of EAs is analysed in depth to conclude that the typical binary classification of the target functions into separable/non-separable and unimodal/multimodal is too general, and characterizing the EAs' response in these terms is misleading. Consequently, more detailed features of the fitness landscape in terms of separability and modality are proposed here and their relevance in the EAs' behaviour is shown through experimentation using standardized benchmark functions that are described using those features. Three different EAs, the genetic algorithm, the Covariance Matrix Adaptation Evolution Strategy and Differential Evolution, are evaluated over these benchmarks and their behaviour is explained in terms of the proposed features.

[1]  Richard J. Duro,et al.  Application domain study of evolutionary algorithms in optimization problems , 2008, GECCO '08.

[2]  Jesús Marín,et al.  How landscape ruggedness influences the performance of real-coded algorithms: a comparative study , 2012, Soft Comput..

[3]  Victor J. Rayward-Smith,et al.  Fitness Distance Correlation and Ridge Functions , 1998, PPSN.

[4]  Bin Li,et al.  Understand behavior and performance of Real Coded Optimization Algorithms via NK-linkage model , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[5]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[6]  Colin R. Reeves,et al.  An Experimental Design Perspective on Genetic Algorithms , 1994, FOGA.

[7]  Ponnuthurai Nagaratnam Suganthan,et al.  Benchmark Functions for the CEC'2013 Special Session and Competition on Large-Scale Global Optimization , 2008 .

[8]  Thomas Bäck,et al.  An analysis of the behavior of simplified evolutionary algorithms on trap functions , 2003, IEEE Trans. Evol. Comput..

[9]  Saku Kukkonen,et al.  Real-parameter optimization with differential evolution , 2005, 2005 IEEE Congress on Evolutionary Computation.

[10]  Carlos A. Coello Coello,et al.  A comparative study of differential evolution variants for global optimization , 2006, GECCO.

[11]  Terry Jones,et al.  Fitness Distance Correlation as a Measure of Problem Difficulty for Genetic Algorithms , 1995, ICGA.

[12]  Yuval Davidor,et al.  Epistasis Variance: A Viewpoint on GA-Hardness , 1990, FOGA.

[13]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[14]  Oliver Kramer,et al.  Surrogate Constraint Functions for CMA Evolution Strategies , 2009, KI.

[15]  Yuval Davidor,et al.  Epistasis Variance: Suitability of a Representation to Genetic Algorithms , 1990, Complex Syst..

[16]  Bart Naudts,et al.  A comparison of predictive measures of problem difficulty in evolutionary algorithms , 2000, IEEE Trans. Evol. Comput..

[17]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[18]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[19]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[20]  Josselin Garnier,et al.  Efficiency of Local Search with Multiple Local Optima , 2001, SIAM J. Discret. Math..

[21]  He-sheng Tang,et al.  Differential evolution strategy for structural system identification , 2008 .

[22]  Kenneth de Jong,et al.  Evolutionary computation: a unified approach , 2007, GECCO.

[23]  Anan Nimtawat,et al.  Automated layout design of beam-slab floors using a genetic algorithm , 2009 .

[24]  Hung-Chang Liao,et al.  The genetic algorithm for breast tumor diagnosis - The case of DNA viruses , 2009, Appl. Soft Comput..

[25]  Ivanoe De Falco,et al.  Differential Evolution as a viable tool for satellite image registration , 2008, Appl. Soft Comput..

[26]  Francisco Herrera,et al.  Tackling Real-Coded Genetic Algorithms: Operators and Tools for Behavioural Analysis , 1998, Artificial Intelligence Review.

[27]  Andrew M. Sutton,et al.  Differential evolution and non-separability: using selective pressure to focus search , 2007, GECCO '07.

[28]  Julian Francis Miller,et al.  Information Characteristics and the Structure of Landscapes , 2000, Evolutionary Computation.

[29]  L. Darrell Whitley,et al.  Nonlinearity, Hyperplane Ranking and the Simple Genetic Algorithm , 1996, Foundations of Genetic Algorithms.

[30]  Jing J. Liang,et al.  Problem Deflnitions and Evaluation Criteria for the CEC 2006 Special Session on Constrained Real-Parameter Optimization , 2006 .

[31]  Weinberger,et al.  Local properties of Kauffman's N-k model: A tunably rugged energy landscape. , 1991, Physical review. A, Atomic, molecular, and optical physics.

[32]  Alain Hertz,et al.  A Taxonomy of Evolutionary Algorithms in Combinatorial Optimization , 1999, J. Heuristics.

[33]  Colin R. Reeves,et al.  Epistasis in Genetic Algorithms: An Experimental Design Perspective , 1995, ICGA.

[34]  M. Schoenauer,et al.  On functions with a given fitness-distance relation , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[35]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[36]  Stefan Droste,et al.  A rigorous analysis of the compact genetic algorithm for linear functions , 2006, Natural Computing.

[37]  Marco Locatelli,et al.  On the Multilevel Structure of Global Optimization Problems , 2005, Comput. Optim. Appl..

[38]  Alejandro Paz-Lopez,et al.  JEAF: A Java Evolutionary Algorithm Framework , 2010, IEEE Congress on Evolutionary Computation.

[39]  Raymond Ros,et al.  A Simple Modification in CMA-ES Achieving Linear Time and Space Complexity , 2008, PPSN.

[40]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[41]  E. Weinberger,et al.  Correlated and uncorrelated fitness landscapes and how to tell the difference , 1990, Biological Cybernetics.

[42]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[43]  P. Stadler,et al.  Random field models for fitness landscapes , 1999 .

[44]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[45]  Chun-Liang Lin,et al.  Structure-specified IIR filter and control design using real structured genetic algorithm , 2009, Appl. Soft Comput..

[46]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[47]  Amit Konar,et al.  Differential Evolution Using a Neighborhood-Based Mutation Operator , 2009, IEEE Transactions on Evolutionary Computation.

[48]  Christian Igel,et al.  Efficient covariance matrix update for variable metric evolution strategies , 2009, Machine Learning.

[49]  Xiaodong Li,et al.  Benchmark Functions for the CEC'2010 Special Session and Competition on Large-Scale , 2009 .

[50]  L. Kallel,et al.  How to detect all maxima of a function , 2001 .

[51]  M. Clergue,et al.  GA-hard functions built by combination of Trap functions , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[52]  Xin Yao,et al.  A Note on Problem Difficulty Measures in Black-Box Optimization: Classification, Realizations and Predictability , 2007, Evolutionary Computation.

[53]  Riccardo Poli,et al.  Information landscapes and problem hardness , 2005, GECCO '05.

[54]  Richard J. Duro,et al.  Real-Valued Multimodal Fitness Landscape Characterization for Evolution , 2010, ICONIP.

[55]  Daniela Zaharie,et al.  Influence of crossover on the behavior of Differential Evolution Algorithms , 2009, Appl. Soft Comput..

[56]  Mohamed Slimane,et al.  A Critical and Empirical Study of Epistasis Measures for Predicting GA Performances: A Summary , 1997, Artificial Evolution.

[57]  Sébastien Vérel,et al.  Fitness landscapes and graphs: multimodularity, ruggedness and neutrality , 2009, GECCO '09.

[58]  Bogdan Filipic,et al.  Tuning EPR spectral parameters with a genetic algorithm , 2001, Appl. Soft Comput..

[59]  Richard J. Duro,et al.  Hydrodynamic Design of Control Surfaces for Ships Using a MOEA with Neuronal Correction , 2009, HAIS.

[60]  Devavrat Shah,et al.  Computing separable functions via gossip , 2005, PODC '06.

[61]  Francisco Herrera,et al.  A taxonomy for the crossover operator for real‐coded genetic algorithms: An experimental study , 2003, Int. J. Intell. Syst..

[62]  Lee Altenberg,et al.  Fitness Distance Correlation Analysis: An Instructive Counterexample , 1997, ICGA.

[63]  Zbigniew Michalewicz,et al.  Genetic algorithms + data structures = evolution programs (2nd, extended ed.) , 1994 .

[64]  L. Darrell Whitley,et al.  The dispersion metric and the CMA evolution strategy , 2006, GECCO.

[65]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[66]  R. Saravanan,et al.  Evolutionary multi criteria design optimization of robot grippers , 2009, Appl. Soft Comput..

[67]  B. Naudts,et al.  Epistasis and Deceptivity , 1999 .