Quasi-Newton Methods, Motivation and Theory

This paper is an attempt to motivate and justify quasi-Newton methods as useful modifications of Newton''s method for general and gradient nonlinear systems of equations. References are given to ample numerical justification; here we give an overview of many of the important theoretical results and each is accompanied by sufficient discussion to make the results and hence the methods plausible. Key Words and Phrases: unconstrained minimization, nonlinear simultaneous equations, update methods, quasi-Newton methods.

[1]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[2]  A. Goldstein.  On Steepest Descent , 1965 .

[3]  J. H. Wilkinson.  The algebraic eigenvalue problem , 1966 .

[4]  C. G. Broyden.  A Class of Methods for Solving Nonlinear Simultaneous Equations , 1965 .

[5]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[6]  S. Goldfeld,et al.  Maximization by Quadratic Hill-Climbing , 1966 .

[7]  Anthony V. Fiacco,et al.  Nonlinear programming;: Sequential unconstrained minimization techniques , 1968 .

[8]  J. D. Pearson.  ON VARIABLE METRIC METHODS OF MINIMIZATION , 1968 .

[9]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .

[10]  J. Daniel.  On the approximate minimization of functionals , 1969 .

[11]  Lenhart K. Schubert.  Modification of a quasi-Newton method for nonlinear equations with a sparse Jacobian , 1970 .

[12]  C. G. Broyden.  The convergence of single-rank quasi-Newton methods , 1970 .

[13]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[14]  D. Shanno.  Conditioning of Quasi-Newton Methods for Function Minimization , 1970 .

[15]  J. Greenstadt.  Variations on Variable-Metric Methods , 1970 .

[16]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[17]  P. Wolfe.  Convergence Conditions for Ascent Methods. II: Some Corrections , 1971 .

[18]  M. Powell.  On the Convergence of the Variable Metric Algorithm , 1971 .

[19]  C. G. Broyden,et al.  The convergence of an algorithm for solving sparse nonlinear systems , 1971 .

[20]  Janusz S. Kowalik,et al.  Iterative methods for nonlinear optimization problems , 1972 .

[21]  L. Dixon.  Variable metric algorithms: Necessary and sufficient conditions for identical behavior of nonquadratic functions , 1972 .

[22]  A. Gourlay,et al.  Rank-one and Rank-two Corrections to Positive Definite Matrices Expressed in Product Form , 1973 .

[23]  R. Brent.  Some Efficient Algorithms for Solving Systems of Nonlinear Equations , 1973 .

[24]  John E. Dennis,et al.  On the Local and Superlinear Convergence of Quasi-Newton Methods , 1973 .

[25]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[26]  W. Burmeister.  Die Konvergenzordnung des Fletcher‐Powell‐Algorithmus , 1973 .

[27]  Philip E. Gill,et al.  Newton-type methods for unconstrained and linearly constrained optimization , 1974, Math. Program..

[28]  Josef Stoer,et al.  Über die Konvergenzordnung Gewisser Rang-2-Verfahren zur Minimierung von Funktionen , 1974 .

[29]  J. J. Moré,et al.  On the Global Convergence of Broyden''s Method , 1974 .

[30]  W. Murray.  Numerical Methods for Unconstrained Optimization , 1975 .

[31]  Josef Stoer.  On the convergence rate of imperfect minimization algorithms in Broyden'sβ-class , 1975, Math. Program..

[32]  M. Powell.  CONVERGENCE PROPERTIES OF A CLASS OF MINIMIZATION ALGORITHMS , 1975 .

[33]  William C. Davidon,et al.  Optimally conditioned optimization algorithms without line searches , 1975, Math. Program..