A New Analysis on the Barzilai-Borwein Gradient Method

Due to its simplicity and efficiency, the Barzilai and Borwein (BB) gradient method has received various attentions in different fields. This paper presents a new analysis of the BB method for two-dimensional strictly convex quadratic functions. The analysis begins with the assumption that the gradient norms at the first two iterations are fixed. We show that there is a superlinear convergence step in at most three consecutive steps. Meanwhile, we provide a better convergence relation for the BB method. The influence of the starting point and the condition number to the convergence rate is comprehensively addressed.

[1]  H. Akaike.  On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method , 1959 .

[2]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[3]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[4]  M. Raydan.  On the Barzilai and Borwein choice of steplength for the gradient method , 1993 .

[5]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[6]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[7]  Jorge Nocedal,et al.  On the Behavior of the Gradient Norm in the Steepest Descent Method , 2002, Comput. Optim. Appl..

[8]  L. Liao,et al.  R-linear convergence of the Barzilai and Borwein gradient method , 2002 .

[9]  Luca Zanni,et al.  Gradient projection methods for quadratic programs and applications in training support vector machines , 2005, Optim. Methods Softw..

[10]  Roger Fletcher,et al.  On the asymptotic behaviour of some new gradient methods , 2005, Math. Program..

[11]  Roger Fletcher,et al.  New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds , 2006, Math. Program..

[12]  Ángel Santos-Palomo,et al.  Updating and downdating an upper trapezoidal sparse orthogonal factorization , 2006 .

[13]  Stephen J. Wright,et al.  Sparse reconstruction by separable approximation , 2009, IEEE Trans. Signal Process..