论文信息 - Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions

Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions

The Nelder--Mead simplex algorithm, first published in 1965, is an enormously popular direct search method for multidimensional unconstrained minimization. Despite its widespread use, essentially no theoretical results have been proved explicitly for the Nelder--Mead algorithm. This paper presents convergence properties of the Nelder--Mead algorithm applied to strictly convex functions in dimensions 1 and 2. We prove convergence to a minimizer for dimension 1, and various limited convergence results for dimension 2. A counterexample of McKinnon gives a family of strictly convex functions in two dimensions and a set of initial conditions for which the Nelder--Mead algorithm converges to a nonminimizer. It is not yet known whether the Nelder--Mead method can be proved to converge to a minimizer for a more specialized class of convex functions in two dimensions.

[1] John A. Nelder,et al. A Simplex Method for Function Minimization , 1965, Comput. J..

[2] N. S. Barnett,et al. Private communication , 1969 .

[3] Daniel John Woods,et al. An interactive approach for solving multi-objective optimization problems (interactive computer, nelder-mead simplex algorithm, graphics) , 1985 .

[4] V. J. Torczoit,et al. Multidirectional search: a direct search algorithm for parallel machines , 1989 .

[5] John E. Dennis,et al. Multidirectional search: a direct search algorithm for parallel machines , 1989 .

[6] J. Dennis,et al. Direct Search Methods on Parallel Machines , 1991 .

[7] A. R. Humphries,et al. Dynamical Systems And Numerical Analysis , 1996 .

[8] Margaret H. Wright,et al. Direct search methods: Once scorned, now respectable , 1996 .

[9] Virginia Torczon,et al. On the Convergence of Pattern Search Algorithms , 1997, SIAM J. Optim..

[10] K. I. M. McKinnon,et al. Convergence of the Nelder-Mead Simplex Method to a Nonstationary Point , 1998, SIAM J. Optim..

[11] F. Walters. Sequential Simplex Optimization - An Update , 1999 .

[12] Paul Tseng,et al. Fortified-Descent Simplicial Search Method: A General Approach , 1999, SIAM J. Optim..

[13] C. T. Kelley,et al. Detection and Remediation of Stagnation in the Nelder--Mead Algorithm Using a Sufficient Decrease Condition , 1999, SIAM J. Optim..