A Survey of Quasi-Newton Equations and Quasi-Newton Methods for Optimization

Quasi-Newton equations play a central role in quasi-Newton methods for optimization and various quasi-Newton equations are available. This paper gives a survey on these quasi-Newton equations and studies properties of quasi-Newton methods with updates satisfying different quasi-Newton equations. These include single-step quasi-Newton equations that use only gradient information and that use both gradient and function value information in one step, and multi-step quasi-Newton equations that use the gradient information in last m steps. Main properties of quasi-Newton methods with updates satisfying different quasi-Newton equations are studied. These properties include the finite termination property, invariance, heredity of positive definite updates, consistency of search directions, global convergence and local superlinear convergence properties.

[1]  J. Nocedal,et al.  Global Convergence of a Class of Quasi-newton Methods on Convex Problems, Siam Some Global Convergence Properties of a Variable Metric Algorithm for Minimization without Exact Line Searches, Nonlinear Programming, Edited , 1996 .

[2]  J. Nocedal,et al.  A tool for the analysis of Quasi-Newton methods with application to unconstrained minimization , 1989 .

[3]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[4]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[5]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[6]  Issam A. R. Moghrabi,et al.  Alternative parameter choices for multi-step Quasi-Newton methods , 1993 .

[7]  T. M. Williams,et al.  Practical Methods of Optimization. Vol. 1: Unconstrained Optimization , 1980 .

[8]  William C. Davidon,et al.  Optimally conditioned optimization algorithms without line searches , 1975, Math. Program..

[9]  M. Powell.  On the Convergence of the Variable Metric Algorithm , 1971 .

[10]  L. C W. Dixon,et al.  Quasi-newton algorithms generate identical points , 1972, Math. Program..

[11]  Jianzhon Zhang,et al.  Properties and numerical performance of quasi-Newton methods with modified quasi-Newton equations , 2001 .

[12]  Issam A. R. Moghrabi,et al.  Multi-step quasi-Newton methods for optimization , 1994 .

[13]  H. Y. Huang.  Unified approach to quadratically convergent algorithms for function minimization , 1970 .

[14]  Miguel F. Anjos.  A Modified Broyden Update with Interpolation , 1993, SIAM J. Sci. Comput..

[15]  E. Spedicato.  A class of rank-one positive definite qnasi-newton updates for unconstrained minimization 2 , 1983 .

[16]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[17]  Jingfeng Zhang,et al.  New Quasi-Newton Equation and Related Methods for Unconstrained Optimization , 1999 .

[18]  John E. Dennis,et al.  On the Local and Superlinear Convergence of Quasi-Newton Methods , 1973 .

[19]  W. Davidon.  Conic Approximations and Collinear Scalings for Optimizers , 1980 .

[20]  Ya-Xiang Yuan,et al.  A modified BFGS algorithm for unconstrained optimization , 1991 .

[21]  R. Fletcher.  Practical Methods of Optimization , 1988 .