A new class of nonlinear conjugate gradient coefficients with global convergence properties

Abstract Nonlinear conjugate gradient (CG) methods have played an important role in solving large-scale unconstrained optimization. Their wide application in many fields is due to their low memory requirements and global convergence properties. Numerous studies and modifications have been conducted recently to improve this method. In this paper, a new class of conjugate gradient coefficients ( β k ) that possess global convergence properties is presented. The global convergence result is established using exact line searches. Numerical result shows that the proposed formula is superior and more efficient when compared to other CG coefficients.

[1]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[2]  Radosław Pytlak,et al.  Conjugate Gradient Algorithms in Nonconvex Optimization , 2008 .

[3]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[4]  N. Andrei.  Accelerated conjugate gradient algorithm with finite difference Hessian/vector product approximation for unconstrained optimization , 2009 .

[5]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[6]  M. J. D. Powell,et al.  Restart procedures for the conjugate gradient method , 1977, Math. Program..

[7]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[8]  A. Goldstein.  On Steepest Descent , 1965 .

[9]  D. Touati-Ahmed,et al.  Efficient hybrid conjugate gradient techniques , 1990 .

[10]  L. Armijo.  Minimization of functions having Lipschitz continuous first partial derivatives. , 1966 .

[11]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[12]  Gonglin Yuan,et al.  A Line Search Algorithm for Unconstrained Optimization , 2010, J. Softw. Eng. Appl..

[13]  Zhi-feng Dai,et al.  Two modified HS type conjugate gradient methods for unconstrained optimization problems , 2011 .

[14]  Xiwen Lu,et al.  A conjugate gradient method with descent direction for unconstrained optimization , 2009, J. Comput. Appl. Math..

[15]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[16]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[17]  Jiapu Zhang,et al.  Global Convergence of Conjugate Gradient Methods without Line Search , 2001, Ann. Oper. Res..

[18]  Zhiwei Xu,et al.  The convergence of conjugate gradient method with nonmonotone line search , 2010, Appl. Math. Comput..

[19]  J. M. Martínez,et al.  A Spectral Conjugate Gradient Method for Unconstrained Optimization , 2001 .

[20]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[21]  K. E. Hillstrom,et al.  A Simulation Test Approach to the Evaluation of Nonlinear Optimization Algorithms , 1977, TOMS.

[22]  Zengxin Wei,et al.  New line search methods for unconstrained optimization , 2009 .

[23]  Jinhua Guo,et al.  A new family of conjugate gradient methods , 2009 .

[24]  M. Powell.  Nonconvex minimization calculations and the conjugate gradient method , 1984 .

[25]  Neculai Andrei,et al.  An Unconstrained Optimization Test Functions Collection , 2008 .

[26]  Guoyin Li,et al.  New nonlinear conjugate gradient formulas for large-scale unconstrained optimization problems , 2006, Appl. Math. Comput..

[27]  Neculai Andrei,et al.  Open Problems in Nonlinear Conjugate Gradient Algorithms for Unconstrained Optimization , 2011 .

[28]  M. Powell.  Convergence properties of algorithms for nonlinear optimization , 1986 .

[29]  Yuhong Dai.  Nonlinear Conjugate Gradient Methods , 2011 .

[30]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[31]  Ya-xiang,et al.  A NOTE ON THE NONLINEAR CONJUGATE GRADIENT METHOD , 2002 .

[32]  Mustafa Mamat,et al.  Comparative Study of Conjugate Gradient Coefficient for Unconstrained Optimization , 2011 .

[33]  P. Wolfe.  Convergence Conditions for Ascent Methods. II , 1969 .