Sizing and least-change secant methods

Oren and Luenberger introduced in 1974 a strategy for replacing Hessian approximations by their scalar multiples and then performing quasi-Newton updates, generally least-change secant updates such as the BFGS or DFP updates [Oren and Luenberger, Management Sci., 20 (1974), pp. 845–862]. In this paper, the function \[\omega (A) = \left( {\frac{{{{{\operatorname{trace}}(A)} / n}}}{{{\operatorname{det}}(A)^{{1 / n}} }}} \right)\] is shown to be a measure of change with a direct connection to the Oren–Luenberger strategy. This measure is interesting because it is related to the $\ell_2$ condition number, but it takes all the eigenvalues of A into account rather than just the extremes. If the class of possible updates is restricted to the Broyden class, i.e., scalar premultiples are not allowed, then the optimal update depends on the dimension of the problem. It may, or may not, be in the convex class, but it becomes the BFGS update as the dimension increases. This seems to be yet another explanation for why ...

[1]  Olvi L. Mangasarian,et al.  Nonlinear Programming , 1969 .

[2]  S. Hoshino A Formulation of Variable Metric Methods , 1972 .

[3]  D. Luenberger,et al.  Self-Scaling Variable Metric (SSVM) Algorithms , 1974 .

[4]  Josef Stoer On the convergence rate of imperfect minimization algorithms in Broyden'sβ-class , 1975, Math. Program..

[5]  William C. Davidon,et al.  Optimally conditioned optimization algorithms without line searches , 1975, Math. Program..

[6]  Shmuel S. Oren,et al.  Optimal conditioning of self-scaling variable Metric algorithms , 1976, Math. Program..

[7]  Robert B. Schnabel,et al.  Analyzing and Improving Quasi-Newton Methods for Unconstrained Optimization , 1977 .

[8]  D. Shanno,et al.  Numerical comparison of several variable-metric algorithms , 1978 .

[9]  Jorge J. Moré,et al.  User Guide for Minpack-1 , 1980 .

[10]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[11]  John E. Dennis,et al.  An Adaptive Nonlinear Least-Squares Algorithm , 1977, TOMS.

[12]  Gene H. Golub,et al.  Matrix computations , 1983 .

[13]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[14]  Narendra Karmarkar,et al.  A new polynomial-time algorithm for linear programming , 1984, STOC '84.

[15]  M. J. D. Powell,et al.  How bad are the BFGS and DFP methods when the objective function is quadratic? , 1986, Math. Program..

[16]  J. Nocedal,et al.  Global Convergence of a Class of Quasi-newton Methods on Convex Problems, Siam Some Global Convergence Properties of a Variable Metric Algorithm for Minimization without Exact Line Searches, Nonlinear Programming, Edited , 1996 .

[17]  J. Nocedal,et al.  A tool for the analysis of Quasi-Newton methods with application to unconstrained minimization , 1989 .

[18]  Roger Fletcher,et al.  A New Variational Result for Quasi-Newton Formulae , 1991, SIAM J. Optim..

[19]  Henry Wolkowicz,et al.  Measures for Symmetric Rank-One Updates , 1994, Math. Oper. Res..