A Numerical Evaluation of Several Stochastic Algorithms on Selected Continuous Global Optimization Test Problems

There is a need for a methodology to fairly compare and present evaluation study results of stochastic global optimization algorithms. This need raises two important questions of (i) an appropriate set of benchmark test problems that the algorithms may be tested upon and (ii) a methodology to compactly and completely present the results. To address the first question, we compiled a collection of test problems, some are better known than others. Although the compilation is not exhaustive, it provides an easily accessible collection of standard test problems for continuous global optimization. Five different stochastic global optimization algorithms have been tested on these problems and a performance profile plot based on the improvement of objective function values is constructed to investigate the macroscopic behavior of the algorithms. The paper also investigates the microscopic behavior of the algorithms through quartile sequential plots, and contrasts the information gained from these two kinds of plots. The effect of the length of run is explored by using three maximum numbers of function evaluations and it is shown to significantly impact the behavior of the algorithms.

[1]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[2]  M. M. Ali,et al.  Topographical Differential Evolution Using Pre-calculated Differentials , 2002 .

[3]  Robert L. Smith,et al.  Improving Hit-and-Run for global optimization , 1993, J. Glob. Optim..

[4]  W. Price.  Global optimization by controlled random search , 1983 .

[5]  Christian Jansson,et al.  A branch and bound algorithm for bound constrained optimization problems without derivatives , 1995, J. Glob. Optim..

[6]  A. Griewank.  Generalized descent for global optimization , 1981 .

[7]  Leo Breiman,et al.  A deterministic algorithm for global optimization , 1993, Math. Program..

[8]  Zelda B. Zabinsky,et al.  Implementation of Stochastic Adaptive Search with Hit-and-Run as a Generator , 2002 .

[9]  C. Storey,et al.  Application of Stochastic Global Optimization Algorithms to Practical Problems , 1997 .

[10]  R. Salomon.  Re-evaluating genetic algorithm performance under coordinate rotation of benchmark functions. A survey of some theoretical and practical aspects of genetic algorithms. , 1996, Bio Systems.

[11]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[12]  M. Montaz Ali,et al.  Population set-based global optimization algorithms: some modifications and numerical studies , 2004, Comput. Oper. Res..

[13]  Zelda B. Zabinsky,et al.  Comparative Assessment of Algorithms and Software for Global Optimization , 2005, J. Glob. Optim..

[14]  Luca Maria Gambardella,et al.  Results of the first international contest on evolutionary optimisation (1st ICEO) , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[15]  Heinz Mühlenbein,et al.  The parallel genetic algorithm as function optimizer , 1991, Parallel Comput..

[16]  A. V. Levy,et al.  The Tunneling Algorithm for the Global Minimization of Functions , 1985 .

[17]  Robert L. Smith,et al.  Efficient Monte Carlo Procedures for Generating Points Uniformly Distributed over Bounded Regions , 1984, Oper. Res..

[18]  D. J. Bell,et al.  Numerical Methods for Unconstrained Optimization , 1979 .

[19]  H. Zimmermann.  Towards global optimization 2: L.C.W. DIXON and G.P. SZEGÖ (eds.) North-Holland, Amsterdam, 1978, viii + 364 pages, US $ 44.50, Dfl. 100,-. , 1979 .

[20]  W. Murray.  Numerical Methods for Unconstrained Optimization , 1975 .

[21]  D. J. Evans,et al.  A sparse preconditioner for symmetric positive definite banded circulant and toeplitz linear systems , 1994 .

[22]  Zelda B. Zabinsky,et al.  Stochastic Adaptive Search , 2002 .

[23]  Zbigniew Michalewicz,et al.  Genetic Algorithms + Data Structures = Evolution Programs , 1996, Springer Berlin Heidelberg.

[24]  Hans-Paul Schwefel,et al.  Evolution and Optimum Seeking: The Sixth Generation , 1993 .

[25]  F. Aluffi-Pentini,et al.  Global optimization and stochastic differential equations , 1985 .

[26]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[27]  Hans-Paul Schwefel,et al.  Evolution and optimum seeking , 1995, Sixth-generation computer technology series.

[28]  W. Price.  Global optimization algorithms for a CAD workstation , 1987 .

[29]  G. McCormick.  Nonlinear Programming: Theory, Algorithms and Applications , 1983 .

[30]  Robert L. Smith,et al.  Simulated annealing for constrained global optimization , 1994, J. Glob. Optim..

[31]  Panos M. Pardalos,et al.  A Collection of Test Problems for Constrained Global Optimization Algorithms , 1990, Lecture Notes in Computer Science.

[32]  K. K. Benke,et al.  A Direct Search Algorithm for Global Optimisation of Multivariate Functions , 1991, Aust. Comput. J..

[33]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[34]  C. Storey,et al.  Modified controlled random search algorithms , 1994 .

[35]  M. E. Johnson,et al.  Generalized simulated annealing for function optimization , 1986 .

[36]  Aimo A. Törn,et al.  Global Optimization , 1999, Science.

[37]  Wyn L. Price,et al.  A Controlled Random Search Procedure for Global Optimisation , 1977, Comput. J..

[38]  Zelda B. Zabinsky,et al.  Global optimization of composite laminates using improving hit and run , 1992 .

[39]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[40]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[41]  David Mautner Himmelblau,et al.  Applied Nonlinear Programming , 1972 .

[42]  Emile H. L. Aarts,et al.  Global optimization and simulated annealing , 1991, Math. Program..