The divergence of the BFGS and Gauss Newton methods

We present examples of divergence for the BFGS and Gauss Newton methods. These examples have objective functions with bounded level sets and other properties concerning the examples published recently in this journal, like unit steps and convexity along the search lines. As these other examples, the iterates, function values and gradients in the new examples fit into the general formulation in our previous work Mascarenhas (Comput Appl Math 26(1), 2007), which also presents an example of divergence for Newton’s method.

[1]  Robert E. Mahony,et al.  Convergence of the Iterates of Descent Methods for Analytic Cost Functions , 2005, SIAM J. Optim..

[2]  L. B. Rall.  A Comparison of the Existence Theorems of Kantorovich and Moore , 1980 .

[3]  Émile Cotton.  Sur les solutions asymptotiques des équations différentielles , 1911 .

[4]  Charles Fefferman,et al.  A Generalized Sharp Whitney Theorem for Jets , 2005 .

[5]  H. Poincaré,et al.  Les méthodes nouvelles de la mécanique céleste , 1899 .

[6]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[7]  Walter F. Mascarenhas.  Newton’s iterates can converge to non-stationary points , 2008, Math. Program..

[8]  J. Ortega.  The Newton-Kantorovich Theorem , 1968 .

[9]  H. Whitney.  Analytic Extensions of Differentiable Functions Defined in Closed Sets , 1934 .

[10]  Stephen J. Wright,et al.  Numerical Optimization (Springer Series in Operations Research and Financial Engineering) , 2000 .

[11]  M. Powell.  Nonconvex minimization calculations and the conjugate gradient method , 1984 .

[12]  Yu-Hong Dai,et al.  Convergence Properties of the BFGS Algoritm , 2002, SIAM J. Optim..

[13]  Walter F. Mascarenhas,et al.  The BFGS method with exact line searches fails for non-convex objective functions , 2004, Math. Program..

[14]  Yu-Hong Dai,et al.  A perfect example for the BFGS method , 2013, Math. Program..

[15]  K. Kurdyka.  On gradients of functions definable in o-minimal structures , 1998 .

[16]  W. Mascarenhas.  On the divergence of line search methods , 2007 .