Algorithm 851: CG_DESCENT, a conjugate gradient method with guaranteed descent

Recently, a new nonlinear conjugate gradient scheme was developed which satisfies the descent condition <b>g</b><sup>T</sup><sub><i>k</i></sub><b>d</b><sub><i>k</i></sub> ≤ −7/8 ‖<b>g</b><sub><i>k</i></sub>‖<sup>2</sup> and which is globally convergent whenever the line search fulfills the Wolfe conditions. This article studies the convergence behavior of the algorithm; extensive numerical tests and comparisons with other methods for large-scale unconstrained optimization are given.

[1]  David F. Shanno,et al.  Remark on “Algorithm 500: Minimization of Unconstrained Multivariate Functions [E4]” , 1980, TOMS.

[2]  G An.  Convergence of Perry and Shanno's Memoryless Quasi-NewtonMethod for Nonconvex Optimization Problems , 1997 .

[3]  M. Powell.  Convergence properties of algorithms for nonlinear optimization , 1986 .

[4]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[5]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[6]  Claude Lemaréchal,et al.  A view of line-searches , 1981 .

[7]  A. Goldstein.  On Steepest Descent , 1965 .

[8]  J. J. Moré,et al.  Newton's Method , 1982 .

[9]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[10]  David F. Shanno.  Globally convergent conjugate gradient algorithms , 1985, Math. Program..

[11]  James Demmel,et al.  Applied Numerical Linear Algebra , 1997 .

[12]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[13]  David G. Wilson,et al.  2. Constrained Optimization , 2005 .

[14]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[15]  刘光辉,et al.  TWO FUNDAMENTAL CONVERGENCE THEOREMS FOR NONLINEAR CONJUGATE GRADIENT METHODS AND THEIR APPLICATIONS , 2001 .

[16]  J. Nocedal.  Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[17]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[18]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[19]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[20]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[21]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[22]  R. Fletcher,et al.  An efficient line search for nonlinear least squares , 1986 .

[23]  A. I. Cohen.  Rate of convergence of several conjugate gradient algorithms. , 1972 .

[24]  Yu-Hong Dai.  A family of hybrid conjugate gradient methods for unconstrained optimization , 2003, Math. Comput..

[25]  W. Hager.  Applied Numerical Linear Algebra , 1987 .

[26]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[27]  M. J. D. Powell,et al.  Restart procedures for the conjugate gradient method , 1977, Math. Program..

[28]  Ya-Xiang Yuan,et al.  An Efficient Hybrid Conjugate Gradient Method for Unconstrained Optimization , 2001, Ann. Oper. Res..

[29]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[30]  W. W. Hager~f.  A Derivative-based Bracketing Scheme for Univariate Minimization and the Conjugate Gradient Method , 2022 .

[31]  D. Shanno.  On the Convergence of a New Conjugate Gradient Algorithm , 1978 .

[32]  P. Wolfe.  Convergence Conditions for Ascent Methods. II: Some Corrections , 1971 .

[33]  J. Daniel.  The Conjugate Gradient Method for Linear and Nonlinear Operator Equations , 1967 .

[34]  David J. Thuente,et al.  Line search algorithms with guaranteed sufficient decrease , 1994, TOMS.

[35]  M. Powell.  Nonconvex minimization calculations and the conjugate gradient method , 1984 .

[36]  L. Liao,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[37]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .

[38]  T. M. Williams,et al.  Practical Methods of Optimization. Vol. 1: Unconstrained Optimization , 1980 .

[39]  Han Jiye,et al.  Two fundamental convergence theorems for nonlinear conjugate gradient methods and their applications , 2001 .

[40]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[41]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[42]  Gene H. Golub,et al.  Some History of the Conjugate Gradient and Lanczos Algorithms: 1948-1976 , 1989, SIAM Rev..

[43]  A. Perry.  A Class of Conjugate Gradient Algorithms with a Two-Step Variable Metric Memory , 1977 .