Extended Barzilai-Borwein method for unconstrained minimization problems

In 1988, Barzilai and Borwein presented a new choice of step size for the gradient method for solving unconstrained minimization problems. Their method aimed to accelerate the convergence of the steepest descent method. The Barzilai-Borwein method requires few storage locations and inexpensive computations. Therefore, several authors have paid attention to the Barzilai-Borwein method and have proposed some variants to solve large-scale unconstrained minimization problems. In this paper, we extend the Barzilai-Borwein method and establish global and Qsuperlinear convergence properties of the proposed method for minimizing a strictly convex quadratic function. Furthermore, we discuss an application of our method to general objective functions. Finally, some numerical experiments are given.

[1]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[2]  L. Grippo,et al.  A truncated Newton method with nonmonotone line search for unconstrained optimization , 1989 .

[3]  W. Hager,et al.  The cyclic Barzilai-–Borwein method for unconstrained optimization , 2006 .

[4]  Jorge J. Moré,et al.  Testing Unconstrained Optimization Software , 1981, TOMS.

[5]  M. Raydan.  On the Barzilai and Borwein choice of steplength for the gradient method , 1993 .

[6]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[7]  J. M. Martínez,et al.  Gradient Method with Retards and Generalizations , 1998 .

[8]  Luigi Grippo,et al.  Nonmonotone Globalization Techniques for the Barzilai-Borwein Gradient Method , 2002, Comput. Optim. Appl..

[9]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[10]  L. Liao,et al.  R-linear convergence of the Barzilai and Borwein gradient method , 2002 .

[11]  DaiYuhong,et al.  A NONMONOTONE CONJUGATE GRADIENT ALGORITHM FOR UNCONSTRAINED OPTIMIZATION , 2002 .

[12]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[13]  Roger Fletcher,et al.  On the Barzilai-Borwein Method , 2005 .

[14]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[15]  Yuhong Dai.  On the Nonmonotone Line Search , 2002 .

[16]  Ya-Xiang Yuan,et al.  Analysis of monotone gradient methods , 2005 .

[17]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .