Metaheuristic vs. deterministic global optimization algorithms: The univariate case

Many practical problems involve the search for the global extremum in the space of the system parameters. The functions to be optimized are often highly multiextremal, black-box with unknown analytical representations, and hard to evaluate even in the case of one parameter to be adjusted in the presence of non-linear constraints. The interest of both the stochastic (in particular, metaheuristic) and mathematical programming (in particular, deterministic) communities to the comparison of metaheuristic and deterministic classes of methods is well recognized. Although both the communities have a huge number of journal and proceedings papers, a few of them are really dedicated to a systematic comparison of the methods belonging to these two classes. This paper meets the requirement of such a comparison between nature-inspired metaheuristic and deterministic algorithms (more than 125,000 launches of the methods have been performed) and presents an attempt (beneficial to practical fields including engineering design) to bring together two rather disjoint communities of metaheuristic and mathematical programming researchers and applied users.

[1]  Marco Gaviano,et al.  A global minimization algorithm for Lipschitz functions , 2007, Optim. Lett..

[2]  Vladimir A. Grishagin,et al.  Adaptive nested optimization scheme for multidimensional global search , 2016, J. Glob. Optim..

[3]  Stefano Lucidi,et al.  A partition-based global optimization algorithm , 2010, J. Glob. Optim..

[4]  Roman G. Strongin,et al.  Introduction to Global Optimization Exploiting Space-Filling Curves , 2013 .

[5]  Yaroslav D. Sergeyev,et al.  An Information Global Optimization Algorithm with Local Tuning , 1995, SIAM J. Optim..

[6]  Victor P. Gergel,et al.  Parallel global optimization on GPU , 2016, Journal of Global Optimization.

[7]  V. A. Grishagin,et al.  Sequential and parallel algorithms for global optimization , 1994 .

[8]  Yaroslav D. Sergeyev,et al.  On the Least-Squares Fitting of Data by Sinusoids , 2016, Advances in Stochastic and Deterministic Global Optimization.

[9]  A. A. Zhigli︠a︡vskiĭ,et al.  Stochastic Global Optimization , 2007 .

[10]  Yaroslav D. Sergeyev,et al.  Deterministic approaches for solving practical black-box global optimization problems , 2015, Adv. Eng. Softw..

[11]  Yaroslav D. Sergeyev,et al.  Derivative-Free Local Tuning and Local Improvement Techniques Embedded in the Univariate Global Optimization , 2016, J. Optim. Theory Appl..

[12]  Mikhail Posypkin,et al.  A deterministic approach to global box-constrained optimization , 2012, Optimization Letters.

[13]  Y. D. Sergeyev,et al.  Global Optimization with Non-Convex Constraints - Sequential and Parallel Algorithms (Nonconvex Optimization and its Applications Volume 45) (Nonconvex Optimization and Its Applications) , 2000 .

[14]  Antanas Zilinskas,et al.  On similarities between two models of global optimization: statistical models and radial basis functions , 2010, J. Glob. Optim..

[15]  Yaroslav D. Sergeyev,et al.  Lipschitz global optimization methods in control problems , 2013, Autom. Remote. Control..

[16]  Julius Zilinskas,et al.  Globally-biased Disimpl algorithm for expensive global optimization , 2014, Journal of Global Optimization.

[17]  Yaroslav D. Sergeyev,et al.  Deterministic Global Optimization: An Introduction to the Diagonal Approach , 2017 .

[18]  M. Locatelli.  Simulated Annealing Algorithms for Continuous Global Optimization , 2002 .

[19]  Lonnie Hamm,et al.  GLOBAL OPTIMIZATION METHODS , 2002 .

[20]  Y. Sergeyev,et al.  Operational zones for comparing metaheuristic and deterministic one-dimensional global optimization algorithms , 2017, Math. Comput. Simul..

[21]  Dervis Karaboga,et al.  A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm , 2007, J. Glob. Optim..

[22]  Yaroslav D. Sergeyev,et al.  Algorithm 829: Software for generation of classes of test functions with known local and global minima for global optimization , 2003, TOMS.

[23]  Helio J. C. Barbosa,et al.  An adaptive constraint handling technique for differential evolution with dynamic use of variants in engineering optimization , 2011 .

[24]  D. Famularo,et al.  Test Problems for Lipschitz Univariate Global Optimization with Multiextremal Constraints , 2002 .

[25]  D. Kvasov,et al.  Lipschitz optimization methods for fitting a sum of damped sinusoids to a series of observations , 2017 .

[26]  Graham Kendall,et al.  Search Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques , 2013 .

[27]  Mauro Brunato,et al.  Reactive Search and Intelligent Optimization , 2008 .

[28]  Y. Sergeyev,et al.  Univariate geometric Lipschitz global optimization algorithms , 2012 .

[29]  János D. Pintér,et al.  Global Optimization: Software, Test Problems, and Applications , 2002 .

[30]  J D Pinter,et al.  Global Optimization in Action—Continuous and Lipschitz Optimization: Algorithms, Implementations and Applications , 2010 .

[31]  Xin-She Yang,et al.  Engineering Optimization: An Introduction with Metaheuristic Applications , 2010 .

[32]  Yaroslav D. Sergeyev,et al.  Index branch-and-bound algorithm for Lipschitz univariate global optimization with multiextremal constraints , 2001, J. Glob. Optim..

[33]  Harish Garg,et al.  A hybrid PSO-GA algorithm for constrained optimization problems , 2016, Appl. Math. Comput..

[34]  Kathryn A. Dowsland,et al.  Simulated Annealing , 1989, Encyclopedia of GIS.

[35]  Yaroslav D. Sergeyev,et al.  Global Search Based on Efficient Diagonal Partitions and a Set of Lipschitz Constants , 2006, SIAM J. Optim..

[36]  Yaroslav D. Sergeyev,et al.  A one-dimensional local tuning algorithm for solving GO problems with partially defined constraints , 2007, Optim. Lett..

[37]  V. Ya. Modorskii,et al.  Optimization in design of scientific products for purposes of cavitation problems , 2016 .

[38]  Christine A. Shoemaker,et al.  Global Convergence of Radial Basis Function Trust-Region Algorithms for Derivative-Free Optimization , 2013, SIAM Rev..

[39]  Vladimir A. Grishagin,et al.  Parallel Characteristical Algorithms for Solving Problems of Global Optimization , 1997, J. Glob. Optim..

[40]  Yaroslav D. Sergeyev,et al.  Deterministic Global Optimization , 2017 .

[41]  S. A. Piyavskii.  An algorithm for finding the absolute extremum of a function , 1972 .

[42]  Yaroslav D. Sergeyev,et al.  Acceleration of Univariate Global Optimization Algorithms Working with Lipschitz Functions and Lipschitz First Derivatives , 2013, SIAM J. Optim..

[43]  Dmitri E. Kvasov.  Multidimensional Lipschitz global optimization based on efficient diagonal partitions , 2008, 4OR.

[44]  Anatoly Zhigljavsky,et al.  Stochastic algorithms for solving structured low-rank matrix approximation problems , 2015, Commun. Nonlinear Sci. Numer. Simul..

[45]  R. Storn,et al.  Differential Evolution: A Practical Approach to Global Optimization (Natural Computing Series) , 2005 .

[46]  Nikolaos V. Sahinidis,et al.  Derivative-free optimization: a review of algorithms and comparison of software implementations , 2013, J. Glob. Optim..

[47]  Dervis Karaboga,et al.  A comparative study of Artificial Bee Colony algorithm , 2009, Appl. Math. Comput..

[48]  Pasquale Daponte,et al.  Two methods for solving optimization problems arising in electronic measurements and electrical engineering , 1999, SIAM J. Optim..

[49]  J. Mockus,et al.  The Bayesian approach to global optimization , 1989 .

[50]  Xin-She Yang,et al.  Firefly Algorithm: Recent Advances and Applications , 2013, ArXiv.