Analysis of a self-scaling quasi-Newton method

We study the self-scaling BFGS method of Oren and Luenberger (1974) for solving unconstrained optimization problems. For general convex functions, we prove that the method is globally convergent with inexact line searches. We also show that the directions generated by the self-scaling BFGS method approach Newton's direction asymptotically. This would ensure superlinear convergence if, in addition, the search directions were well-scaled, but we show that this is not always the case. We find that the method has a major drawback: to achieve superlinear convergence it may be necessary to evaluate the function twice per iteration, even very near the solution. An example is constructed to show that the step-sizes required to achieve a superlinear rate converge to 2 and 0.5 alternately.

[1]  H. Piaggio.  Mathematical Analysis , 1955, Nature.

[2]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .

[3]  J. D. Pearson.  Variable metric methods of minimisation , 1969, Comput. J..

[4]  P. Wolfe.  Convergence Conditions for Ascent Methods. II: Some Corrections , 1971 .

[5]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[6]  D. Luenberger,et al.  Self-Scaling Variable Metric (SSVM) Algorithms , 1974 .

[7]  D. F. Shanno,et al.  Matrix conditioning and nonlinear optimization , 1978, Math. Program..

[8]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[9]  S. Oren.  Perspectives on self-scaling variable metric algorithms , 1982 .

[10]  David G. Luenberger,et al.  Linear and nonlinear programming , 1984 .

[11]  J. Nocedal,et al.  Global Convergence of a Class of Quasi-newton Methods on Convex Problems, Siam Some Global Convergence Properties of a Variable Metric Algorithm for Minimization without Exact Line Searches, Nonlinear Programming, Edited , 1996 .

[12]  M. J. D. Powell,et al.  Updating conjugate directions by the BFGS formula , 1987, Math. Program..

[13]  J. Nocedal,et al.  A tool for the analysis of Quasi-Newton methods with application to unconstrained minimization , 1989 .

[14]  Jorge Nocedal,et al.  On the Behavior of Broyden's Class of Quasi-Newton Methods , 1992, SIAM J. Optim..

[15]  J. Dennis,et al.  Sizing and least-change secant methods , 1993 .

[16]  Jorge Nocedal,et al.  Automatic Column Scaling Strategies for Quasi-Newton Methods , 1993, SIAM J. Optim..

[17]  Dirk Siegel,et al.  Modifying the BFGS update by a new column scaling technique , 1994, Math. Program..