Two families of scaled three-term conjugate gradient methods with sufficient descent property for nonconvex optimization

In this paper, we present two families of modified three-term conjugate gradient methods for solving unconstrained large-scale smooth optimization problems. We show that our new families satisfy the Dai-Liao conjugacy condition and the sufficient descent condition under any line search technique which guarantees the positiveness of y k T s k ${y_{k}^{T}} s_{k}$ . For uniformly convex functions, we indicate that our families are globally convergent under weak-Wolfe-Powell line search technique and standard conditions on the objective function. We also establish a weaker global convergence theorem for general smooth functions under similar assumptions. Our numerical experiments for 260 standard problems and seven other recently developed conjugate gradient methods illustrate that the members of our families are numerically efficient and effective.

[1]  Yanlin Wu A modified three-term PRP conjugate gradient algorithm for optimization models , 2017, Journal of inequalities and applications.

[2]  Neculai Andrei,et al.  Accelerated scaled memoryless BFGS preconditioned conjugate gradient algorithm for unconstrained optimization , 2010, Eur. J. Oper. Res..

[3]  Hongwei Liu,et al.  New version of the three-term conjugate gradient method based on spectral scaling conjugacy condition that generates descent search direction , 2015, Appl. Math. Comput..

[4]  Ya-Xiang Yuan,et al.  A Nonlinear Conjugate Gradient Method with a Strong Global Convergence Property , 1999, SIAM J. Optim..

[5]  J. M. Martínez,et al.  A Spectral Conjugate Gradient Method for Unconstrained Optimization , 2001 .

[6]  Hongwei Liu,et al.  Several efficient gradient methods with approximate optimal stepsizes for large scale unconstrained optimization , 2018, J. Comput. Appl. Math..

[7]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[8]  C. M. Reeves,et al.  Function minimization by conjugate gradients , 1964, Comput. J..

[9]  Hiroshi Yabe,et al.  A family of three-term conjugate gradient methods with sufficient descent property for unconstrained optimization , 2015, Comput. Optim. Appl..

[10]  Saman Babaie-Kafaki,et al.  On optimality of two adaptive choices for the parameter of Dai–Liao method , 2016, Optim. Lett..

[11]  Neculai Andrei,et al.  On three-term conjugate gradient algorithms for unconstrained optimization , 2013, Appl. Math. Comput..

[12]  Masoud Fatemi An Optimal Parameter for Dai–Liao Family of Conjugate Gradient Methods , 2016, J. Optim. Theory Appl..

[13]  Li Zhang,et al.  Some descent three-term conjugate gradient methods and their global convergence , 2007, Optim. Methods Softw..

[14]  N. Andrei A modified Polak–Ribière–Polyak conjugate gradient algorithm for unconstrained optimization , 2011 .

[15]  L. M. Zou,et al.  Some three-term conjugate gradient methods with the inexact line search condition , 2018, Calcolo.

[16]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[17]  Neculai Andrei,et al.  Accelerated adaptive Perry conjugate gradient algorithms based on the self-scaling memoryless BFGS update , 2017, J. Comput. Appl. Math..

[18]  Bing Zheng,et al.  Two New Dai–Liao-Type Conjugate Gradient Methods for Unconstrained Optimization Problems , 2017, J. Optim. Theory Appl..

[19]  M. J. D. Powell,et al.  Restart procedures for the conjugate gradient method , 1977, Math. Program..

[20]  Masoud Fatemi A scaled conjugate gradient method for nonlinear unconstrained optimization , 2017, Optim. Methods Softw..

[21]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[22]  Shengwei Yao,et al.  An adaptive three-term conjugate gradient method based on self-scaling memoryless BFGS matrix , 2018, J. Comput. Appl. Math..

[23]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[24]  F. Mezzadri,et al.  An inexact Newton method for solving complementarity problems in hydrodynamic lubrication , 2018 .

[25]  Wah June Leong,et al.  A scaled three-term conjugate gradient method for unconstrained optimization , 2016, Journal of Inequalities and Applications.

[26]  Hiroshi Yabe,et al.  A Three-Term Conjugate Gradient Method with Sufficient Descent Property for Unconstrained Optimization , 2011, SIAM J. Optim..

[27]  S. J. Li,et al.  New three-term conjugate gradient method with guaranteed global convergence , 2014, Int. J. Comput. Math..

[28]  E. Polak,et al.  Note sur la convergence de méthodes de directions conjuguées , 1969 .

[29]  L. Nazareth A conjugate direction algorithm without line searches , 1977 .

[30]  Weijun Zhou,et al.  A descent modified Polak–Ribière–Polyak conjugate gradient method and its global convergence , 2006 .

[31]  Boris Polyak The conjugate gradient method in extremal problems , 1969 .

[32]  William W. Hager,et al.  A New Conjugate Gradient Method with Guaranteed Descent and an Efficient Line Search , 2005, SIAM J. Optim..

[33]  Yuting Chen,et al.  A subspace conjugate gradient algorithm for large-scale unconstrained optimization , 2017, Numerical Algorithms.

[34]  Neculai Andrei,et al.  A simple three-term conjugate gradient algorithm for unconstrained optimization , 2013, J. Comput. Appl. Math..

[35]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[36]  Neculai Andrei,et al.  An Unconstrained Optimization Test Functions Collection , 2008 .

[37]  Boris Polyak The conjugate gradient method in extreme problems , 2015 .

[38]  Mustafa Mamat,et al.  An efficient modified Polak–Ribière–Polyak conjugate gradient method with global convergence properties , 2017, Optim. Methods Softw..

[39]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[40]  Reza Ghanbari,et al.  A descent family of Dai–Liao conjugate gradient methods , 2014, Optim. Methods Softw..

[41]  Y. -H. Dai,et al.  New Conjugacy Conditions and Related Nonlinear Conjugate Gradient Methods , 2001 .

[42]  J. Burke,et al.  Line Search Methods for Convex-Composite Optimization , 2018, 1806.05218.

[43]  Zhong Wan,et al.  A three-term conjugate gradient algorithm for large-scale unconstrained optimization problems , 2015 .

[44]  L. Dixon,et al.  A new three-term conjugate gradient method , 1985 .

[45]  Jin-kui Liu,et al.  New three-term conjugate gradient method for solving unconstrained optimization problems , 2014 .