Since CEC 2005 competition on real-parameter optimisation: a decade of research, progress and comparative analysis’s weakness

Real-parameter optimisation is a prolific research line with hundreds of publications per year. There exists an impressive number of alternatives in both algorithm families and enhancements over their respective original proposals. In this work, we analyse if this growth in the number of publications is correlated with a real progress in the field. We have selected five approaches from one of the most significant journals in the field and compared them with the winner of the competition celebrated within the IEEE Congress on Evolutionary Computation 2005. We observe that not only these methods are unable to get the good results of the winner of the competition, published several years before, but that they often avoid this type of comparison. Instead, they usually compare with other approaches from the same family. We conclude that the comparison with the state-of-the-art of the field should be mandatory to promote a real progress and to prevent that the area becomes obfuscated for outsiders.

[1]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[2]  J. Ford,et al.  Hybrid estimation of distribution algorithm for global optimization , 2004 .

[3]  Richard M. Friedberg,et al.  A Learning Machine: Part I , 1958, IBM J. Res. Dev..

[4]  Anne Auger,et al.  COCO: a platform for comparing continuous optimizers in a black-box setting , 2016, Optim. Methods Softw..

[5]  Jing J. Liang,et al.  Comprehensive learning particle swarm optimizer for global optimization of multimodal functions , 2006, IEEE Transactions on Evolutionary Computation.

[6]  Yuping Wang,et al.  An orthogonal genetic algorithm with quantization for global numerical optimization , 2001, IEEE Trans. Evol. Comput..

[7]  Anne Auger,et al.  Performance evaluation of an advanced local search evolutionary algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[8]  Liyan Zhang,et al.  Empirical study of particle swarm optimizer with an increasing inertia weight , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[9]  Jing J. Liang,et al.  Novel composition test functions for numerical global optimization , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[10]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[11]  Andy J. Keane,et al.  Meta-Lamarckian learning in memetic algorithms , 2004, IEEE Transactions on Evolutionary Computation.

[12]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[13]  Xin-She Yang,et al.  Test Functions for Global Optimization : A Comprehensive Survey , 2013 .

[14]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[15]  Jun Zhang,et al.  Orthogonal Learning Particle Swarm Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[16]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[17]  Nikolaus Hansen,et al.  Benchmarking of Continuous Black Box Optimization Algorithms , 2012, Evolutionary Computation.

[18]  Adam P. Piotrowski,et al.  Regarding the rankings of optimization heuristics based on artificially-constructed benchmark functions , 2015, Inf. Sci..

[19]  Janez Brest,et al.  Self-Adapting Control Parameters in Differential Evolution: A Comparative Study on Numerical Benchmark Problems , 2006, IEEE Transactions on Evolutionary Computation.

[20]  Thomas Stützle,et al.  A Note on Bound Constraints Handling for the IEEE CEC’05 Benchmark Function Suite , 2014, Evolutionary Computation.

[21]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[22]  José Neves,et al.  The fully informed particle swarm: simpler, maybe better , 2004, IEEE Transactions on Evolutionary Computation.

[23]  Russell C. Eberhart,et al.  Tracking and optimizing dynamic systems with particle swarms , 2001, Proceedings of the 2001 Congress on Evolutionary Computation (IEEE Cat. No.01TH8546).

[24]  Jouni Lampinen,et al.  A Fuzzy Adaptive Differential Evolution Algorithm , 2005, Soft Comput..

[25]  Thomas Stützle,et al.  Ant Colony Optimization Theory , 2004 .

[26]  Kalyanmoy Deb,et al.  Self-Adaptive Genetic Algorithms with Simulated Binary Crossover , 2001, Evolutionary Computation.

[27]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[28]  Dennis Weyland,et al.  A Rigorous Analysis of the Harmony Search Algorithm: How the Research Community can be Misled by a "Novel" Methodology , 2010, Int. J. Appl. Metaheuristic Comput..

[29]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[30]  Nikolaus Hansen,et al.  Benchmarking a BI-population CMA-ES on the BBOB-2009 function testbed , 2009, GECCO '09.

[31]  Andries Petrus Engelbrecht,et al.  Analysis and classification of optimisation benchmark functions and benchmark suites , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[32]  R. Eberhart,et al.  Empirical study of particle swarm optimization , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[33]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[34]  Kalyanmoy Deb,et al.  A Computationally Efficient Evolutionary Algorithm for Real-Parameter Optimization , 2002, Evolutionary Computation.

[35]  Marco Dorigo,et al.  Ant system: optimization by a colony of cooperating agents , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[36]  Konstantinos E. Parsopoulos,et al.  UPSO: A Unified Particle Swarm Optimization Scheme , 2019, International Conference of Computational Methods in Sciences and Engineering 2004 (ICCMSE 2004).

[37]  Luca Maria Gambardella,et al.  Results of the first international contest on evolutionary optimisation (1st ICEO) , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[38]  OVEIS ABEDINIA,et al.  A new metaheuristic algorithm based on shark smell optimization , 2016, Complex..

[39]  Alex S. Fukunaga,et al.  Improving the search performance of SHADE using linear population size reduction , 2014, 2014 IEEE Congress on Evolutionary Computation (CEC).

[40]  Jason Sheng-Hong Tsai,et al.  A self-optimization approach for L-SHADE incorporated with eigenvector-based crossover and successful-parent-selecting framework on CEC 2015 benchmark set , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[41]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[42]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[43]  Maurice Clerc,et al.  The particle swarm - explosion, stability, and convergence in a multidimensional complex space , 2002, IEEE Trans. Evol. Comput..

[44]  Thomas Bäck,et al.  Evolutionary computation: Toward a new philosophy of machine intelligence , 1997, Complex..

[45]  Kevin M. Passino,et al.  Biomimicry of bacterial foraging for distributed optimization and control , 2002 .

[46]  Carlos A. Coello Coello,et al.  THEORETICAL AND NUMERICAL CONSTRAINT-HANDLING TECHNIQUES USED WITH EVOLUTIONARY ALGORITHMS: A SURVEY OF THE STATE OF THE ART , 2002 .

[47]  Amit Konar,et al.  Differential Evolution Using a Neighborhood-Based Mutation Operator , 2009, IEEE Transactions on Evolutionary Computation.

[48]  Martin Middendorf,et al.  A hierarchical particle swarm optimizer and its adaptive variant , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[49]  László Pál,et al.  A Comparison of Global Search Algorithms for Continuous Black Box Optimization , 2012, Evolutionary Computation.

[50]  Shinn-Ying Ho,et al.  OPSO: Orthogonal Particle Swarm Optimization and Its Application to Task Assignment Problems , 2008, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[51]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[52]  Zong Woo Geem,et al.  A New Heuristic Optimization Algorithm: Harmony Search , 2001, Simul..

[53]  Xiaodong Li,et al.  Designing benchmark problems for large-scale continuous optimization , 2015, Inf. Sci..

[54]  F. Glover HEURISTICS FOR INTEGER PROGRAMMING USING SURROGATE CONSTRAINTS , 1977 .

[55]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[56]  Lawrence J. Fogel,et al.  Artificial Intelligence through Simulated Evolution , 1966 .

[57]  Ioan Cristian Trelea,et al.  The particle swarm optimization algorithm: convergence analysis and parameter selection , 2003, Inf. Process. Lett..

[58]  Saman K. Halgamuge,et al.  Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients , 2004, IEEE Transactions on Evolutionary Computation.

[59]  Jing J. Liang,et al.  Dynamic multi-swarm particle swarm optimizer , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[60]  Lalit M. Patnaik,et al.  Adaptive probabilities of crossover and mutation in genetic algorithms , 1994, IEEE Trans. Syst. Man Cybern..

[61]  Xin Yao,et al.  Evolutionary programming made faster , 1999, IEEE Trans. Evol. Comput..

[62]  Kalyan Veeramachaneni,et al.  Fitness-distance-ratio based particle swarm optimization , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[63]  Andries Petrus Engelbrecht,et al.  Self-adaptive Differential Evolution , 2005, CIS.

[64]  Xin Yao,et al.  Making a Difference to Differential Evolution , 2008, Advances in Metaheuristics for Hard Optimization.

[65]  Alex S. Fukunaga,et al.  Success-history based parameter adaptation for Differential Evolution , 2013, 2013 IEEE Congress on Evolutionary Computation.

[66]  Francisco Herrera,et al.  A taxonomy for the crossover operator for real‐coded genetic algorithms: An experimental study , 2003, Int. J. Intell. Syst..

[67]  Jirí Kubalík,et al.  Experimental Comparison of Six Population-Based Algorithms for Continuous Black Box Optimization , 2012, Evolutionary Computation.

[68]  Robert G. Reynolds,et al.  An ensemble sinusoidal parameter adaptation incorporated with L-SHADE for solving CEC2014 benchmark problems , 2016, 2016 IEEE Congress on Evolutionary Computation (CEC).

[69]  Bernardetta Addis,et al.  A new class of test functions for global optimization , 2007, J. Glob. Optim..

[70]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[71]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[72]  Xin-She Yang,et al.  A literature survey of benchmark functions for global optimisation problems , 2013, Int. J. Math. Model. Numer. Optimisation.

[73]  Jing J. Liang,et al.  Novel benchmark functions for continuous multimodal optimization with comparative results , 2016, Swarm Evol. Comput..

[74]  Francisco Herrera,et al.  A study on the use of non-parametric tests for analyzing the evolutionary algorithms’ behaviour: a case study on the CEC’2005 Special Session on Real Parameter Optimization , 2009, J. Heuristics.

[75]  Kenneth Sörensen,et al.  Metaheuristics - the metaphor exposed , 2015, Int. Trans. Oper. Res..

[76]  K Krishnakumar,et al.  Solving large parameter optimization problems using genetic algorithms , 1995 .

[77]  John H. Holland,et al.  Outline for a Logical Theory of Adaptive Systems , 1962, JACM.

[78]  Hans-Paul Schwefel,et al.  Numerical Optimization of Computer Models , 1982 .

[79]  Pinaki Mazumder,et al.  SAGA : a unification of the genetic algorithm with simulated annealing and its application to macro-cell placement , 1994, Proceedings of 7th International Conference on VLSI Design.

[80]  George E. P. Box,et al.  Evolutionary Operation: a Method for Increasing Industrial Productivity , 1957 .

[81]  Francisco Herrera,et al.  A Walk into Metaheuristics for Engineering Optimization: Principles, Methods and Recent Trends , 2015, Int. J. Comput. Intell. Syst..

[82]  Francisco J. Rodríguez,et al.  Arbitrary function optimisation with metaheuristics , 2012, Soft Comput..

[83]  J. Snyman A new and dynamic method for unconstrained minimization , 1982 .

[84]  Shu-Mei Guo,et al.  Enhancing Differential Evolution Utilizing Eigenvector-Based Crossover Operator , 2015, IEEE Transactions on Evolutionary Computation.

[85]  Zhi Li,et al.  Genetic Algorithm That Considers Scattering for THz Quantitative Analysis , 2015, IEEE Transactions on Terahertz Science and Technology.

[86]  Xiaodong Li,et al.  A framework for generating tunable test functions for multimodal optimization , 2011, Soft Comput..

[87]  Yongling Zheng,et al.  On the convergence analysis and parameter selection in particle swarm optimization , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[88]  Nikolaus Hansen,et al.  A restart CMA evolution strategy with increasing population size , 2005, 2005 IEEE Congress on Evolutionary Computation.

[89]  Xin Yao,et al.  Evolutionary programming using mutations based on the Levy probability distribution , 2004, IEEE Transactions on Evolutionary Computation.

[90]  Thomas Stützle,et al.  Performance evaluation of automatically tuned continuous optimizers on different benchmark sets , 2015, Appl. Soft Comput..

[91]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[92]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[93]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[94]  Thomas Stützle,et al.  Frankenstein's PSO: A Composite Particle Swarm Optimization Algorithm , 2009, IEEE Transactions on Evolutionary Computation.

[95]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .