Analysis and classification of optimisation benchmark functions and benchmark suites

New and existing optimisation algorithms are often compared by evaluating their performance on a benchmark suite. This set of functions aims to evaluate the algorithm across a range of problems and serves as a baseline measurement of how the algorithm may perform on real-world problems. It is important that the functions serve as a good representative of commonly occurring problems. In order to select functions that will make up the benchmark suite, the characteristics and relationships among the functions must be known. This paper characterises the landscapes of two commonly used benchmark suites, and uses these landscape characteristics to obtain a high level view of the current state of benchmark functions. This is done by using a self-organising feature map to cluster and analyse functions based on landscape characteristics. It is found that while there are numerous functions that cover a wide range of characteristics, there are characteristics that are under represented, or not even covered at all. Furthermore, it is discovered that common benchmark suites are composed of functions which are highly similar according to the measured characteristics.

[1]  Katherine M. Malan.  Characterising continuous optimisation problems for particle swarm optimisation performance prediction , 2014 .

[2]  L. Darrell Whitley,et al.  The dispersion metric and the CMA evolution strategy , 2006, GECCO.

[3]  Sébastien Vérel,et al.  Where are bottlenecks in NK fitness landscapes? , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[4]  Riccardo Poli,et al.  Information landscapes , 2005, GECCO '05.

[5]  Anne Auger,et al.  Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions , 2009 .

[6]  Vesselin K. Vassilev,et al.  Smoothness, ruggedness and neutrality of fitness landscapes: from theory to application , 2003 .

[7]  Andrew M. Sutton,et al.  PSO and multi-funnel landscapes: how cooperation might limit exploration , 2006, GECCO.

[8]  Andries Petrus Engelbrecht,et al.  Quantifying ruggedness of continuous landscapes using entropy , 2009, 2009 IEEE Congress on Evolutionary Computation.

[9]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[10]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[11]  Andries Petrus Engelbrecht,et al.  A survey of techniques for characterising fitness landscapes and some possible ways forward , 2013, Inf. Sci..

[12]  Terry Jones,et al.  Fitness Distance Correlation as a Measure of Problem Difficulty for Genetic Algorithms , 1995, ICGA.

[13]  Andries Petrus Engelbrecht,et al.  A progressive random walk algorithm for sampling continuous fitness landscapes , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[14]  Andries Petrus Engelbrecht,et al.  Ruggedness, funnels and gradients in fitness landscapes and the effect on PSO performance , 2013, 2013 IEEE Congress on Evolutionary Computation.