Comparative Analysis of Continuous Global Optimization Methods

In this paper we evaluate the performance and compare 19 different heuristics for solving continuous global optimization. They are all based on the following metaheuristics: Simulated annealing, Variable neighborhood search, Particle swarm optimization, and Differential evolution. Codes of methods are taken from their authors. The comparison on usual test instances (convex and non-convex) is performed on the same computer. Dimensions of test functions are changed from 10 to 100, thus effectively covering small and large scale problems. The results measured by computational efforts and ranked statistics show that the recent DE-VNS heuristic outperforms the other 18 algorithms on selected problems. Its better performances are noted in solving non-convex problems.

[1]  M. F. Cardoso,et al.  The simplex-simulated annealing approach to continuous non-linear optimization , 1996 .

[2]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[3]  Syed Murtuza Baker,et al.  Comparison of different algorithms for simultaneous estimation of multiple parameters in kinetic metabolic models , 2010, J. Integr. Bioinform..

[4]  R. Salomon.  Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[5]  Qingfu Zhang,et al.  Differential Evolution With Composite Trial Vector Generation Strategies and Control Parameters , 2011, IEEE Transactions on Evolutionary Computation.

[6]  René Thomsen,et al.  A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[7]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[8]  Samuel Kotz,et al.  A novel extension of the triangular distribution and its parameter estimation , 2002 .

[9]  Arthur C. Sanderson,et al.  JADE: Adaptive Differential Evolution With Optional External Archive , 2009, IEEE Transactions on Evolutionary Computation.

[10]  Mirjana Cangalovic,et al.  General variable neighborhood search for the continuous optimization , 2006, Eur. J. Oper. Res..

[11]  Nenad Mladenovic,et al.  Gaussian variable neighborhood search for continuous optimization , 2011, Comput. Oper. Res..

[12]  Fred Glover,et al.  Tabu Search - Part II , 1989, INFORMS J. Comput..

[13]  Pascal Van Hentenryck,et al.  A simulated annealing approach to the traveling tournament problem , 2006, J. Sched..

[14]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[15]  Josef Tvrdík,et al.  Differential evolution with competitive setting of control parameters , 2007 .

[16]  David Johnson,et al.  The triangular distribution as a proxy for the beta distribution in risk analysis , 1997 .

[17]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[18]  Jean Dickinson Gibbons,et al.  Nonparametric Statistical Inference , 1972, International Encyclopedia of Statistical Science.

[19]  Pierre Hansen,et al.  Variable Neighborhood Search , 2018, Handbook of Heuristics.

[20]  Jim Smith,et al.  Operator and parameter adaptation in genetic algorithms , 1997, Soft Comput..

[21]  W. Kruskal,et al.  Use of Ranks in One-Criterion Variance Analysis , 1952 .

[22]  William H. Press,et al.  Simulated Annealing Optimization over Continuous Spaces , 1991 .

[23]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[24]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[25]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[26]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[27]  Nelson Maculan,et al.  A Function to Test Methods Applied to Global Minimization of Potential Energy of Molecules , 2004, Numerical Algorithms.

[28]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[29]  K. Chandrasekar,et al.  Performance Comparison of GA, DE, PSO and SA Approaches in Enhancement of Total Transfer Capability using FACTS Devices , 2012 .

[30]  Fred W. Glover,et al.  Tabu Search - Part I , 1989, INFORMS J. Comput..

[31]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[32]  P. N. Suganthan,et al.  Differential Evolution Algorithm With Strategy Adaptation for Global Numerical Optimization , 2009, IEEE Transactions on Evolutionary Computation.

[33]  D. Wolfe,et al.  Nonparametric Statistical Methods. , 1974 .

[34]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[35]  Riccardo Poli,et al.  Particle Swarm Optimisation , 2011 .