A Two-Phase Gradient Method for Quadratic Programming Problems with a Single Linear Constraint and Bounds on the Variables

We propose a gradient-based method for quadratic programming problems with a single linear constraint and bounds on the variables. Inspired by the GPCG algorithm for bound-constrained convex quadratic programming [J.J. Mor\'e and G. Toraldo, SIAM J. Optim. 1, 1991], our approach alternates between two phases until convergence: an identification phase, which performs gradient projection iterations until either a candidate active set is identified or no reasonable progress is made, and an unconstrained minimization phase, which reduces the objective function in a suitable space defined by the identification phase, by applying either the conjugate gradient method or a recently proposed spectral gradient method. However, the algorithm differs from GPCG not only because it deals with a more general class of problems, but mainly for the way it stops the minimization phase. This is based on a comparison between a measure of optimality in the reduced space and a measure of bindingness of the variables that are on the bounds, defined by extending the concept of proportioning, which was proposed by some authors for box-constrained problems. If the objective function is bounded, the algorithm converges to a stationary point thanks to a suitable application of the gradient projection method in the identification phase. For strictly convex problems, the algorithm converges to the optimal solution in a finite number of steps even in case of degeneracy. Extensive numerical experiments show the effectiveness of the proposed approach.

[1]  Luca Zanni,et al.  Gradient projection methods for quadratic programs and applications in training support vector machines , 2005, Optim. Methods Softw..

[2]  G. Toraldo,et al.  On spectral properties of steepest descent methods , 2013 .

[3]  Gerardo Toraldo,et al.  On the Solution of Large Quadratic Programming Problems with Bound Constraints , 1991, SIAM J. Optim..

[4]  Wei Guo,et al.  Handling Nonpositive Curvature in a Limited Memory Steepest Descent Method , 2016 .

[5]  J. Navarro-Pedreño Numerical Methods for Least Squares Problems , 1996 .

[6]  Paul H. Calamai,et al.  Projected gradient methods for linearly constrained problems , 1987, Math. Program..

[7]  Valeria Ruggiero,et al.  On the steplength selection in gradient methods for unconstrained optimization , 2018, Appl. Math. Comput..

[8]  Laurent Condat Fast projection onto the simplex and the l1\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\pmb {l}_\mathbf {1}$$\end{ , 2015, Mathematical Programming.

[9]  William W. Hager,et al.  A New Active Set Algorithm for Box Constrained Optimization , 2006, SIAM J. Optim..

[10]  Daniela di Serafino,et al.  On the regularizing behavior of the SDA and SDC gradient methods in the solution of linear ill-posed problems , 2016, J. Comput. Appl. Math..

[11]  K. Schittkowski,et al.  NONLINEAR PROGRAMMING , 2022 .

[12]  W. Marsden I and J , 2012 .

[13]  L. Zanni,et al.  A scaled gradient projection method for constrained image deblurring , 2008 .

[14]  Ya-xiang,et al.  A NEW STEPSIZE FOR THE STEEPEST DESCENT METHOD , 2006 .

[15]  Joachim Schöberl,et al.  Minimizing Quadratic Functions Subject to Bound Constraints with the Rate of Convergence and Finite Termination , 2005, Comput. Optim. Appl..

[16]  Ana Friedlander,et al.  On the numerical solution of bound constrained optimization problems , 1989 .

[17]  Panos M. Pardalos,et al.  Constrained Global Optimization: Algorithms and Applications , 1987, Lecture Notes in Computer Science.

[18]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[19]  Roger Fletcher,et al.  Projected Barzilai-Borwein methods for large-scale box-constrained quadratic programming , 2005, Numerische Mathematik.

[20]  Zdenek Dostál,et al.  Minimizing quadratic functions with semidefinite Hessian subject to bound constraints , 2015, Comput. Math. Appl..

[21]  J. J. Moré,et al.  Quasi-Newton updates with bounds , 1987 .

[22]  Zdenek Dostál,et al.  Box Constrained Quadratic Programming with Proportioning and Projections , 1997, SIAM J. Optim..

[23]  William W. Hager,et al.  An Affine-Scaling Interior-Point Method for Continuous Knapsack Constraints with Application to Support Vector Machines , 2011, SIAM J. Optim..

[24]  G. Toraldo,et al.  On the identification property of a projected gradient method , 1993 .

[25]  Karen Willcox,et al.  Optimal $$L_2$$L2-norm empirical importance weights for the change of probability measure , 2017, Stat. Comput..

[26]  Daniel P. Robinson,et al.  A Solver for Nonconvex Bound-Constrained Quadratic Optimization , 2015, SIAM J. Optim..

[27]  B. Friedlander,et al.  An Adaptive Algorithm for Bound Constrained Quadratic Minimization , 1997 .

[28]  William W. Hager,et al.  An efficient gradient method using the Yuan steplength , 2014, Comput. Optim. Appl..

[29]  B. Parlett The Symmetric Eigenvalue Problem , 1981 .

[30]  Ana Friedlander,et al.  On the Maximization of a Concave Quadratic Function with Box Constraints , 1994, SIAM J. Optim..

[31]  W. Hager,et al.  An active set algorithm for nonlinear optimization with polyhedral constraints , 2016, 1606.01992.

[32]  José Mario Martínez,et al.  Nonmonotone Spectral Projected Gradient Methods on Convex Sets , 1999, SIAM J. Optim..

[33]  Roger Fletcher,et al.  New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds , 2006, Math. Program..

[34]  Valeria Ruggiero,et al.  A note on spectral properties of some gradient methods , 2016 .

[35]  Laurent Condat,et al.  A Fast Projection onto the Simplex and the l 1 Ball , 2015 .

[36]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[37]  J. J. Moré,et al.  Algorithms for bound constrained quadratic programming problems , 1989 .

[38]  V. Vapnik Estimation of Dependences Based on Empirical Data , 2006 .

[39]  L. Zanni,et al.  New adaptive stepsize selections in gradient methods , 2008 .

[40]  Pasumarti V. Kamesam,et al.  Multipoint methods for separable nonlinear networks , 1984 .