Multi-step quasi-Newton methods for optimization

Abstract Quasi-Newton methods update, at each iteration, the existing Hessian approximation (or its inverse) by means of data deriving from the step just completed. We show how “multi-step” methods (employing, in addition, data from previous iterations) may be constructed by means of interpolating polynomials, leading to a generalization of the “secant” (or “quasi-Newton”) equation. The issue of positive-definiteness in the Hessian approximation is addressed and shown to depend on a generalized version of the condition which is required to hold in the original “single-step” methods. The results of extensive numerical experimentation indicate strongly that computational advantages can accrue from such an approach (by comparison with “single-step” methods), particularly as the dimension of the problem increases.

[1]  J. Dennis,et al.  Direct secant updates of matrix factorizations , 1982 .

[2]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[3]  David F. Shanno,et al.  Remark on “Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4]” , 1980, TOMS.

[4]  M. C. Biggs.  Minimization Algorithms Making Use of Non-quadratic Properties of the Objective Function , 1971 .

[5]  G. W. Johnson,et al.  A Quasi-Newton Method Employing Direct Secant Updates of Matrix Factorizations , 1983 .

[6]  F. Soesianto,et al.  On the solution of highly structured nonlinear equations , 1992 .

[7]  John A. Ford,et al.  On the use of function-values in unconstrained optimisation , 1989 .

[8]  David F. Shanno,et al.  Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4] , 1976, TOMS.

[9]  W. Burmeister.  Inversionsfreie Verfahren zur Lösung nichtlinearer Operatorgleichungen , 1972 .

[10]  P. Toint,et al.  On Large Scale Nonlinear Least Squares Calculations , 1987 .

[11]  J. Dennis.  On Some Methods Based on Broyden's Secant Approximation to the Hessian , 1971 .

[12]  J. Ford,et al.  On the construction of minimisation methods of quasi-Newton type , 1987 .

[13]  I. D. Coope,et al.  A conjugate direction implementation of the BFGS algorithm with automatic scaling , 1989, The Journal of the Australian Mathematical Society. Series B. Applied Mathematics.

[14]  John A. Ford,et al.  On the use of curvature estimates in quasi-Newtonian methods , 1991 .

[15]  C. G. Broyden.  A Class of Methods for Solving Nonlinear Simultaneous Equations , 1965 .

[16]  Götz Alefeld,et al.  A QUADRATICALLY CONVERGENT KRAWCZYK-LIKE ALGORITHM* , 1983 .

[17]  P. Toint,et al.  Testing a class of methods for solving minimization problems with simple bounds on the variables , 1988 .

[18]  M. C. Biggs.  A Note on Minimization Algorithms which make Use of Non-quardratic Properties of the Objective Function , 1973 .