The Simplex Gradient and Noisy Optimization Problems

Many classes of methods for noisy optimization problems are based on function information computed on sequences of simplices. The Nelder-Mead, multidirectional search, and implicit filtering methods are three such methods. The performance of these methods can be explained in terms of the difference approximation of the gradient implicit in the function evaluations. Insight can be gained into choice of termination criteria, detection of failure, and design of new methods.

[1]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[2]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[3]  C. G. Broyden.  Quasi-Newton methods and their application to function minimisation , 1967 .

[4]  D. Bertsekas.  On the Goldstein-Levitin-Polyak gradient projection method , 1974, CDC 1974.

[5]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[6]  D. Mayne,et al.  Nondifferential optimization via adaptive smoothing , 1984 .

[7]  Richard H. Byrd,et al.  Parallel quasi-Newton methods for unconstrained optimization , 1988, Math. Program..

[8]  John E. Dennis,et al.  Direct Search Methods on Parallel Machines , 1991, SIAM J. Optim..

[9]  C. T. Kelley,et al.  Doping profiles for optimum class B performance of GaAs MESFET amplifiers , 1991, [1991] Proceedings IEEE/Cornell Conference on Advanced Concepts in High Speed Semiconductor Devices and Circuits.

[10]  C. Kelley,et al.  Simulated performance optimization of GaAs MESFET amplifiers , 1991, [1991] Proceedings IEEE/Cornell Conference on Advanced Concepts in High Speed Semiconductor Devices and Circuits.

[11]  Virginia Torczon,et al.  On the Convergence of the Multidirectional Search Algorithm , 1991, SIAM J. Optim..

[12]  C. Kelley,et al.  Yield optimization using a GaAs process simulator coupled to a physical device model , 1992 .

[13]  Carl Tim Kelley,et al.  An algorithm for optimizing functions with multiple minima , 1993 .

[14]  S. K. Zavriev,et al.  On the global optimization properties of finite-difference local descent algorithms , 1993, J. Glob. Optim..

[15]  P. GILMOREy,et al.  Implicit Filtering and Optimal Design Problems , 1994 .

[16]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[17]  Brian W. Kernighan,et al.  WISE design of indoor wireless systems: practical computation and optimization , 1995 .

[18]  Cass T. Miller,et al.  Implicit Filtering and Optimal Design Problems , 1995 .

[19]  C. T. Kelley,et al.  An Implicit Filtering Algorithm for Optimization of Functions with Many Local Minima , 1995, SIAM J. Optim..

[20]  C. Kelley,et al.  Use of an Implicit Filtering Algorithm for Mechanical System Parameter Identification , 1996 .

[21]  Virginia Torczon,et al.  On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[22]  O. SIAMJ.,et al.  ON THE CONVERGENCE OF PATTERN SEARCH ALGORITHMS , 1997 .

[23]  Carl Tim Kelley,et al.  Optimal design of high speed mechanical systems , 1997 .

[24]  K. I. M. McKinnon,et al.  Convergence of the Nelder-Mead Simplex Method to a Nonstationary Point , 1998, SIAM J. Optim..

[25]  C. T. Kelley,et al.  Detection and Remediation of Stagnation in the Nelder--Mead Algorithm Using a Sufficient Decrease Condition , 1999, SIAM J. Optim..

[26]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .