A modified two-point stepsize gradient algorithm for unconstrained minimization

Based on a modified secant equation proposed by Li and Fukushima, we derive a stepsize for the Barzilai–Borwein gradient method. Then, using the newly proposed stepsize and another effective stepsize proposed by Dai et al. in an adaptive scheme that is based on the objective function convexity, we suggest a modified two-point stepsize gradient algorithm. We also show that the limit point of the sequence generated by our algorithm is first-order critical. Finally, our numerical comparisons done on a set of unconstrained optimization test problems from the CUTEr collection are presented. At first, we compare the performance of our algorithm with two other two-point stepsize gradient algorithms proposed by Dai et al. and Raydan. Then, numerical comparisons between the implementations of our algorithm and two conjugate gradient methods proposed by Gilbert and Nocedal, and Hestenes and Stiefel, and also, with the memoryless BFGS algorithm proposed by Liu and Nocedal, are made. Furthermore, to provide a numerical support for our adaptive approach, we compare two other two-point stepsize gradient algorithms, one of which applies the stepsize proposed by Dai et al. and the other applies our newly proposed stepsize, with our algorithm which applies both of these stepsizes together. Numerical results demonstrating the efficiency of our algorithm, in the sense of the performance profile introduced by Dolan and Moré, are reported.

[1]  Jin Yun Yuan,et al.  Modified Two-Point Stepsize Gradient Methods for Unconstrained Optimization , 2002, Comput. Optim. Appl..

[2]  Ya-Xiang Yuan,et al.  Optimization Theory and Methods: Nonlinear Programming , 2010 .

[3]  M. Fukushima,et al.  A modified BFGS method and its global convergence in nonconvex minimization , 2001 .

[4]  M. Powell.  Nonconvex minimization calculations and the conjugate gradient method , 1984 .

[5]  G. Forsythe.  On the asymptotic directions of thes-dimensional optimum gradient method , 1968 .

[6]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[7]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[8]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[9]  Jingfeng Zhang,et al.  New Quasi-Newton Equation and Related Methods for Unconstrained Optimization , 1999 .

[10]  L. Liao,et al.  R-linear convergence of the Barzilai and Borwein gradient method , 2002 .

[11]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[12]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[13]  Jianguo Liu,et al.  A modified BFGS method and its superlinear convergence in nonconvex minimization with general line search rule , 2008 .

[14]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .

[15]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[16]  Jorge Nocedal,et al.  Global Convergence Properties of Conjugate Gradient Methods for Optimization , 1992, SIAM J. Optim..

[17]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[18]  Roger Fletcher,et al.  On the Barzilai-Borwein Method , 2005 .

[19]  H. Akaike.  On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method , 1959 .

[20]  M. Raydan.  On the Barzilai and Borwein choice of steplength for the gradient method , 1993 .

[21]  Li Zhang,et al.  A nonlinear conjugate gradient method based on the MBFGS secant condition , 2006, Optim. Methods Softw..

[22]  Ya-Xiang Yuan,et al.  A modified BFGS algorithm for unconstrained optimization , 1991 .