Escaping local minima with local derivative-free methods: a numerical investigation

[1]  Warren Hare,et al.  Methods to compare expensive stochastic optimization algorithms with random restarts , 2018, J. Glob. Optim..

[2]  Katya Scheinberg,et al.  Methodologies and software for derivative-free optimization , 2017 .

[3]  Nando de Freitas,et al.  A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning , 2010, ArXiv.

[4]  Coralia Cartis,et al.  A derivative-free Gauss–Newton method , 2017, Mathematical Programming Computation.

[5]  Stefano Lucidi,et al.  Exploiting derivative-free local searches in DIRECT-type algorithms for global optimization , 2016, Comput. Optim. Appl..

[6]  Jaroslav Fowkes,et al.  Bayesian numerical analysis : global optimization and other applications , 2011 .

[7]  Nikolaus Hansen,et al.  Adapting arbitrary normal mutation distributions in evolution strategies: the covariance matrix adaptation , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[8]  Marco Locatelli,et al.  A Note on the Griewank Test Function , 2003, J. Glob. Optim..

[9]  Charles Audet,et al.  Parallel Space Decomposition of the Mesh Adaptive Direct Search Algorithm , 2007, SIAM J. Optim..

[10]  Gianni Di Pillo,et al.  A DIRECT-type approach for derivative-free constrained global optimization , 2016, Comput. Optim. Appl..

[11]  Charles Audet,et al.  OrthoMADS: A Deterministic MADS Instance with Orthogonal Directions , 2008, SIAM J. Optim..

[12]  Charles Audet,et al.  Derivative-Free and Blackbox Optimization , 2017 .

[13]  Arnold Neumaier,et al.  SNOBFIT -- Stable Noisy Optimization by Branch and Fit , 2008, TOMS.

[14]  Charles Audet,et al.  Nonsmooth optimization through Mesh Adaptive Direct Search and Variable Neighborhood Search , 2006, J. Glob. Optim..

[15]  Margaret H. Wright,et al.  Direct search methods: Once scorned, now respectable , 1996 .

[16]  Ameet Talwalkar,et al.  Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization , 2016, J. Mach. Learn. Res..

[17]  Thomas Stützle,et al.  Automatic Algorithm Configuration Based on Local Search , 2007, AAAI.

[18]  Michael Lange,et al.  Automated Tiling of Unstructured Mesh Computations with Application to Seismological Modeling , 2017, ACM Trans. Math. Softw..

[19]  Nikolaus Hansen,et al.  Completely Derandomized Self-Adaptation in Evolution Strategies , 2001, Evolutionary Computation.

[20]  David D. Cox,et al.  Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures , 2013, ICML.

[21]  C. D. Perttunen,et al.  Lipschitzian optimization without the Lipschitz constant , 1993 .

[22]  Jing J. Liang,et al.  Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real-Parameter Optimization , 2005 .

[23]  Charles Audet,et al.  Mesh Adaptive Direct Search Algorithms for Constrained Optimization , 2006, SIAM J. Optim..

[24]  Stefan M. Wild,et al.  Benchmarking Derivative-Free Optimization Algorithms , 2009, SIAM J. Optim..

[25]  Marco Sciandrone,et al.  A Derivative-Free Algorithm for Bound Constrained Optimization , 2002, Comput. Optim. Appl..

[26]  Stefano Lucidi,et al.  An algorithmic framework based on primitive directions and nonmonotone line searches for black-box optimization problems with integer variables , 2020, Math. Program. Comput..

[27]  C. Storey,et al.  Topographical Multilevel Single Linkage , 1994, J. Glob. Optim..

[28]  Katya Scheinberg,et al.  Black-Box Optimization in Machine Learning with Trust Region Based Derivative Free Algorithm , 2017, ArXiv.

[29]  Zelda B. Zabinsky,et al.  A Numerical Evaluation of Several Stochastic Algorithms on Selected Continuous Global Optimization Test Problems , 2005, J. Glob. Optim..

[30]  M. J. D. Powell,et al.  Direct search algorithms for optimization calculations , 1998, Acta Numerica.

[31]  Masha Sosonkina,et al.  Design and implementation of a massively parallel version of DIRECT , 2008, Comput. Optim. Appl..

[32]  Nikolaos V. Sahinidis,et al.  Derivative-free optimization: a review of algorithms and comparison of software implementations , 2013, J. Glob. Optim..

[33]  Aimo A. Törn,et al.  Topographical global optimization using pre-sampled points , 1994, J. Glob. Optim..

[34]  Charles Audet,et al.  Globalization strategies for Mesh Adaptive Direct Search , 2008, Comput. Optim. Appl..

[35]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[36]  Sébastien Le Digabel,et al.  Algorithm xxx : NOMAD : Nonlinear Optimization with the MADS algorithm , 2010 .

[37]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[38]  Charles Audet,et al.  The Mesh Adaptive Direct Search Algorithm for Granular and Discrete Variables , 2018, SIAM J. Optim..

[39]  Julius Zilinskas,et al.  Globally-biased Disimpl algorithm for expensive global optimization , 2014, Journal of Global Optimization.

[40]  Clifford A. Shaffer,et al.  Dynamic Data Structures for a Direct Search Algorithm , 2002, Comput. Optim. Appl..

[41]  C. Shoemaker,et al.  Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization , 2013 .

[42]  Charles Audet,et al.  Reducing the Number of Function Evaluations in Mesh Adaptive Direct Search Algorithms , 2012, SIAM J. Optim..

[43]  G. T. Timmer,et al.  Stochastic global optimization methods part II: Multi level methods , 1987, Math. Program..

[44]  G. T. Timmer,et al.  Stochastic global optimization methods part I: Clustering methods , 1987, Math. Program..

[45]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[46]  Luís N. Vicente,et al.  Direct Multisearch for Multiobjective Optimization , 2011, SIAM J. Optim..

[47]  Stefano Lucidi,et al.  A Linesearch-Based Derivative-Free Approach for Nonsmooth Constrained Optimization , 2014, SIAM J. Optim..

[48]  Jan Mayer,et al.  A numerical evaluation of preprocessing and ILU-type preconditioners for the solution of unsymmetric sparse linear systems using iterative methods , 2009, TOMS.

[49]  Donald R. Jones,et al.  Efficient Global Optimization of Expensive Black-Box Functions , 1998, J. Glob. Optim..

[50]  Charles Audet,et al.  Convergence of Mesh Adaptive Direct Search to Second-Order Stationary Points , 2006, SIAM J. Optim..

[51]  Coralia Cartis,et al.  Improving the Flexibility and Robustness of Model-based Derivative-free Optimization Solvers , 2018, ACM Trans. Math. Softw..

[52]  Kevin Leyton-Brown,et al.  Sequential Model-Based Optimization for General Algorithm Configuration , 2011, LION.

[53]  Nando de Freitas,et al.  Taking the Human Out of the Loop: A Review of Bayesian Optimization , 2016, Proceedings of the IEEE.