Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming

Summary.This paper studies projected Barzilai-Borwein (PBB) methods for large-scale box-constrained quadratic programming. Recent work on this method has modified the PBB method by incorporating the Grippo-Lampariello-Lucidi (GLL) nonmonotone line search, so as to enable global convergence to be proved. We show by many numerical experiments that the performance of the PBB method deteriorates if the GLL line search is used. We have therefore considered the question of whether the unmodified method is globally convergent, which we show not to be the case, by exhibiting a counter example in which the method cycles. A new projected gradient method (PABB) is then considered that alternately uses the two Barzilai-Borwein steplengths. We also give an example in which this method may cycle, although its practical performance is seen to be superior to the PBB method. With the aim of both ensuring global convergence and preserving the good numerical performance of the unmodified methods, we examine other recent work on nonmonotone line searches, and propose a new adaptive variant with some attractive features. Further numerical experiments show that the PABB method with the adaptive line search is the best BB-like method in the positive definite case, and it compares reasonably well against the GPCG algorithm of Moré and Toraldo. In the indefinite case, the PBB method with the adaptive line search is shown on some examples to find local minima with better solution values, and hence may be preferred for this reason.

[1]  H. Akaike.  On a successive transformation of probability distribution and its application to the analysis of the optimum gradient method , 1959 .

[2]  A. Goldstein.  Convex programming in Hilbert space , 1964 .

[3]  Boris Polyak,et al.  Constrained minimization methods , 1966 .

[4]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[5]  D. Bertsekas.  On the Goldstein-Levitin-Polyak gradient projection method , 1974, CDC 1974.

[6]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[7]  Paul H. Calamai,et al.  Projected gradient methods for linearly constrained problems , 1987, Math. Program..

[8]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[9]  J. J. Moré,et al.  Algorithms for bound constrained quadratic programming problems , 1989 .

[10]  Stephen J. Wright.  Implementing proximal point methods for linear programming , 1990 .

[11]  Gerardo Toraldo,et al.  On the Solution of Large Quadratic Programming Problems with Bound Constraints , 1991, SIAM J. Optim..

[12]  Jon W. Tolle,et al.  A class of methods for solving large, convex quadratic programs subject to box constraints , 1991, Math. Program..

[13]  M. Raydan.  On the Barzilai and Borwein choice of steplength for the gradient method , 1993 .

[14]  G. Toraldo,et al.  On the identification property of a projected gradient method , 1993 .

[15]  Ana Friedlander,et al.  On the Maximization of a Concave Quadratic Function with Box Constraints , 1994, SIAM J. Optim..

[16]  Marcos Raydan,et al.  The Barzilai and Borwein Gradient Method for the Large Scale Unconstrained Minimization Problem , 1997, SIAM J. Optim..

[17]  Philippe L. Toint,et al.  Non-monotone trust-region algorithms for nonlinear optimization subject to convex constraints , 1997, Math. Program..

[18]  J. M. Martínez,et al.  Gradient Method with Retards and Generalizations , 1998 .

[19]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[20]  Luigi Grippo,et al.  Nonmonotone Globalization Techniques for the Barzilai-Borwein Gradient Method , 2002, Comput. Optim. Appl..

[21]  L. Liao,et al.  R-linear convergence of the Barzilai and Borwein gradient method , 2002 .

[22]  Jin Yun Yuan,et al.  Modified Two-Point Stepsize Gradient Methods for Unconstrained Optimization , 2002, Comput. Optim. Appl..

[23]  J. M. Martínez,et al.  Inexact spectral projected gradient methods on convex sets , 2003 .

[24]  Luca Zanni,et al.  A parallel solver for large quadratic programs in training support vector machines , 2003, Parallel Comput..

[25]  Yuhong Dai.  Alternate step gradient method , 2003 .

[26]  Valeria Ruggiero,et al.  Variable Projection Methods for Large—Scale Quadratic Optimization in Data Analysis Applications , 2003 .

[27]  Hongchao Zhang,et al.  Adaptive Two-Point Stepsize Gradient Algorithm , 2001, Numerical Algorithms.

[28]  Roger Fletcher,et al.  On the Barzilai-Borwein Method , 2005 .

[29]  Luca Zanni,et al.  Gradient projection methods for quadratic programs and applications in training support vector machines , 2005, Optim. Methods Softw..

[30]  Roger Fletcher,et al.  On the asymptotic behaviour of some new gradient methods , 2005, Math. Program..

[31]  Ángel Santos-Palomo,et al.  Updating and downdating an upper trapezoidal sparse orthogonal factorization , 2006 .