On the relation between quadratic termination and convergence properties of minimization algorithms

SummaryMany algorithms for solving minimization problems of the form $$\mathop {\min }\limits_{x \in R^n } f(x) = f(\bar x),f:R^n \to R,$$ are devised such that they terminate with the optimal solution $$\bar x$$ within at mostn steps, when applied to the minimization of strictly convex quadratic functionsf onRn. In this paper general conditions are given, which together with the quadratic termination property, will ensure that the algorithm locally converges at leastn-step quadratically to a local minimum $$\bar x$$ for sufficiently smooth nonquadratic functionsf. These conditions apply to most algorithms with the quadratic termination property.

[1]  Roger Fletcher,et al.  A Rapidly Convergent Descent Method for Minimization , 1963, Comput. J..

[2]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[3]  J. Daniel.  The Conjugate Gradient Method for Linear and Nonlinear Operator Equations , 1967 .

[4]  C. G. Broyden.  Quasi-Newton methods and their application to function minimisation , 1967 .

[5]  J. D. Pearson.  ON VARIABLE METRIC METHODS OF MINIMIZATION , 1968 .

[6]  J. D. Pearson.  Variable metric methods of minimisation , 1969, Comput. J..

[7]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[8]  H. Y. Huang.  Unified approach to quadratically convergent algorithms for function minimization , 1970 .

[9]  C. G. Broyden.  The Convergence of a Class of Double-rank Minimization Algorithms 1. General Considerations , 1970 .

[10]  R. Fletcher,et al.  A New Approach to Variable Metric Algorithms , 1970, Comput. J..

[11]  M. Powell.  On the Convergence of the Variable Metric Algorithm , 1971 .

[12]  J. Dennis.  Toward a Unified Convergence Theory for Newton-Like Methods , 1971 .

[13]  J. Dennis.  On Some Methods Based on Broyden's Secant Approximation to the Hessian , 1971 .

[14]  Garth P. McCormick,et al.  Methods of conjugate directions versus quasi-Newton methods , 1972, Math. Program..

[15]  A. I. Cohen.  Rate of convergence of several conjugate gradient algorithms. , 1972 .

[16]  Melanie L. Lenard,et al.  Practical convergence conditions for unconstrained optimization , 1973, Math. Program..

[17]  John E. Dennis,et al.  On the Local and Superlinear Convergence of Quasi-Newton Methods , 1973 .

[18]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[19]  W. Burmeister.  Die Konvergenzordnung des Fletcher‐Powell‐Algorithmus , 1973 .

[20]  D. Luenberger,et al.  Self-Scaling Variable Metric (SSVM) Algorithms , 1974 .

[21]  K. Ritter,et al.  Alternative proofs of the convergence properties of the conjugate-gradient method , 1974 .

[22]  Josef Stoer,et al.  Über die Konvergenzordnung Gewisser Rang-2-Verfahren zur Minimierung von Funktionen , 1974 .

[23]  S. Oren.  SELF-SCALING VARIABLE METRIC (SSVM) ALGORITHMS Part II: Implementation and Experiments*t , 1974 .

[24]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[25]  Über das Konvergenzverhalten gewisser Update-Verfahren zur Minimierung einer reellwertigen Funktion bei nichtperfekter Schrittlängenbestimmung , 1975 .

[26]  Josef Stoer.  On the convergence rate of imperfect minimization algorithms in Broyden'sβ-class , 1975, Math. Program..

[27]  Melanie L. LENARD.  Practical convergence conditions for the Davidon-Fletcher-Powell method , 1975, Math. Program..

[28]  Melanie L. Lenard.  Convergence conditions for restarted conjugate gradient methods with inaccurate line searches , 1976, Math. Program..