On a numerical instability of Davidon-like methods

The Davidon-Fletcher-Powell method of function minimization [2], [3] has attained widespread popularity. Yet it goes wrong from time to time. Among the conditions reported are: 1. Broyden [1] states that negative steps had to be taken occasionally. 2. McCormick [5] noted that reinitialization of the matrix every now and then improved the method's performance. 3. Wolfe [6] has reported cases of convergence to nonstationary points. The author has encountered similar behavior in his own work, and has found it invariably the result of the matrix turning singular, due to the cause detailed below. The author believes other workers' difficulties probably originate in the same cause. We wish to find the minimum of a continuously differentiable function F(x) (boldface lower case letters denote vectors). In Davidon's method we proceed iteratively: If xi is the value of x at the ith iteration, then