Two effective hybrid conjugate gradient algorithms based on modified BFGS updates

Based on two modified secant equations proposed by Yuan, and Li and Fukushima, we extend the approach proposed by Andrei, and introduce two hybrid conjugate gradient methods for unconstrained optimization problems. Our methods are hybridizations of Hestenes-Stiefel and Dai-Yuan conjugate gradient methods. Under proper conditions, we show that one of the proposed algorithms is globally convergent for uniformly convex functions and the other is globally convergent for general functions. To enhance the performance of the line search procedure, we propose a new approach for computing the initial value of the steplength for initiating the line search procedure. We give a comparison of the implementations of our algorithms with two efficiently representative hybrid conjugate gradient methods proposed by Andrei using unconstrained optimization test problems from the CUTEr collection. Numerical results show that, in the sense of the performance profile introduced by Dolan and Moré, the proposed hybrid algorithms are competitive, and in some cases more efficient.

[1]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[2]  M. Fukushima,et al.  A modified BFGS method and its global convergence in nonconvex minimization , 2001 .

[3]  W. Hager,et al.  A SURVEY OF NONLINEAR CONJUGATE GRADIENT METHODS , 2005 .

[4]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[5]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[6]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[7]  B. V. Shah,et al.  Integer and Nonlinear Programming , 1971 .

[8]  M. Powell.  Nonconvex minimization calculations and the conjugate gradient method , 1984 .

[9]  C. Storey,et al.  Global convergence result for conjugate gradient methods , 1991 .

[10]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[11]  Jianguo Liu,et al.  A modified BFGS method and its superlinear convergence in nonconvex minimization with general line search rule , 2008 .

[12]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[13]  Yu-Hong Dai.  New properties of a nonlinear conjugate gradient method , 2001, Numerische Mathematik.

[14]  Ya-Xiang Yuan,et al.  Convergence Properties of Nonlinear Conjugate Gradient Methods , 1999, SIAM J. Optim..

[15]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[16]  Guoyin Li,et al.  New conjugacy condition and related new conjugate gradient methods for unconstrained optimization , 2007 .

[17]  Ya-Xiang Yuan,et al.  Optimization theory and methods , 2006 .

[18]  Neculai Andrei,et al.  Accelerated hybrid conjugate gradient algorithm with modified secant condition for unconstrained optimization , 2010, Numerical Algorithms.

[19]  Ya-Xiang Yuan,et al.  A modified BFGS algorithm for unconstrained optimization , 1991 .

[20]  N. Andrei.  Hybrid Conjugate Gradient Algorithm for Unconstrained Optimization , 2009 .

[21]  Dong Wang,et al.  Notes on the Dai-Yuan-Yuan modified spectral gradient method , 2010, J. Comput. Appl. Math..

[22]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[23]  Neculai Andrei,et al.  Another hybrid conjugate gradient algorithm for unconstrained optimization , 2008, Numerical Algorithms.

[24]  Li Zhang,et al.  A nonlinear conjugate gradient method based on the MBFGS secant condition , 2006, Optim. Methods Softw..

[25]  Neculai Andrei,et al.  Scaled conjugate gradient algorithms for unconstrained optimization , 2007, Comput. Optim. Appl..

[26]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[27]  D. Touati-Ahmed,et al.  Efficient hybrid conjugate gradient techniques , 1990 .

[28]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[29]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[30]  Jingfeng Zhang,et al.  New Quasi-Newton Equation and Related Methods for Unconstrained Optimization , 1999 .

[31]  J. M. Martínez,et al.  A Spectral Conjugate Gradient Method for Unconstrained Optimization , 2001 .

[32]  M. Al-Baali.  Descent Property and Global Convergence of the Fletcher—Reeves Method with Inexact Line Search , 1985 .

[33]  C. Storey,et al.  Efficient generalized conjugate gradient algorithms, part 1: Theory , 1991 .

[34]  Ya-Xiang Yuan,et al.  An Efficient Hybrid Conjugate Gradient Method for Unconstrained Optimization , 2001, Ann. Oper. Res..