Primal and dual active-set methods for convex quadratic programming

Computational methods are proposed for solving a convex quadratic program (QP). Active-set methods are defined for a particular primal and dual formulation of a QP with general equality constraints and simple lower bounds on the variables. In the first part of the paper, two methods are proposed, one primal and one dual. These methods generate a sequence of iterates that are feasible with respect to the equality constraints associated with the optimality conditions of the primal–dual form. The primal method maintains feasibility of the primal inequalities while driving the infeasibilities of the dual inequalities to zero. The dual method maintains feasibility of the dual inequalities while moving to satisfy the primal inequalities. In each of these methods, the search directions satisfy a KKT system of equations formed from Hessian and constraint components associated with an appropriate column basis. The composition of the basis is specified by an active-set strategy that guarantees the nonsingularity of each set of KKT equations. Each of the proposed methods is a conventional active-set method in the sense that an initial primal- or dual-feasible point is required. In the second part of the paper, it is shown how the quadratic program may be solved as a coupled pair of primal and dual quadratic programs created from the original by simultaneously shifting the simple-bound constraints and adding a penalty term to the objective function. Any conventional column basis may be made optimal for such a primal–dual pair of shifted-penalized problems. The shifts are then updated using the solution of either the primal or the dual shifted problem. An obvious application of this approach is to solve a shifted dual QP to define an initial feasible point for the primal (or vice versa). The computational performance of each of the proposed methods is evaluated on a set of convex problems from the CUTEst test collection.

[1]  A. Charnes Optimality and Degeneracy in Linear Programming , 1952 .

[2]  G. Dantzig,et al.  Notes on Linear Programming: Part 1. The Generalized Simplex Method for Minimizing a Linear Form under Linear Inequality Restraints , 1954 .

[3]  K. Ritter,et al.  A method for solving nonlinear maximum-problems depending on parameters , 1967 .

[4]  J. Bunch,et al.  Direct Methods for Solving Symmetric Indefinite Systems of Linear Equations , 1971 .

[5]  R. Fletcher A General Quadratic Programming Algorithm , 1971 .

[6]  Paula M. J. Harris Pivot selection methods of the Devex LP code , 1973, Math. Program..

[7]  J. Bunch,et al.  Some stable methods for calculating inertia and solving symmetric linear systems , 1977 .

[8]  R. Fletcher Factorizing symmetric indefinite matrices , 1976 .

[9]  Robert G. Bland,et al.  New Finite Pivoting Rules for the Simplex Method , 1977, Math. Oper. Res..

[10]  Philip E. Gill,et al.  Numerically stable methods for quadratic programming , 1978, Math. Program..

[11]  J. Bunch,et al.  A computational method for the indefinite quadratic programming problem , 1980 .

[12]  K. Ritter On Parametric Linear and Quadratic Programming Problems. , 1981 .

[13]  John K. Reid,et al.  The Multifrontal Solution of Indefinite Sparse Symmetric Linear , 1983, TOMS.

[14]  Donald Goldfarb,et al.  A numerically stable dual method for solving strictly convex quadratic programs , 1983, Math. Program..

[15]  Nicholas I. M. Gould,et al.  A weighted gram-schmidt method for convex quadratic programming , 1984, Math. Program..

[16]  M. Powell On the quadratic programming algorithm of Goldfarb and Idnani , 1985 .

[17]  Stephen C. Hoyle A Single-Phase Method for Quadratic Programming. , 1985 .

[18]  P. Gill,et al.  A Schur-complement method for sparse quadratic programming , 1987 .

[19]  Michael A. Saunders,et al.  A practical anti-cycling procedure for linearly constrained optimization , 1989, Math. Program..

[20]  Michael A. Saunders,et al.  Inertia-Controlling Methods for General Quadratic Programming , 1991, SIAM Rev..

[21]  N. Gould An Algorithm for Large-Scale Quadratic Programming , 1991 .

[22]  W. Murray,et al.  Newton methods for large-scale linear equality-constrained minimization , 1993 .

[23]  R. Fletcher,et al.  Resolving degeneracy in quadratic programming , 1993, Ann. Oper. Res..

[24]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[25]  M. Best An Algorithm for the Solution of the Parametric Quadratic Programming Problem , 1996 .

[26]  Natashia Boland,et al.  A dual-active-set algorithm for positive semi-definite quadratic programming , 1997, Math. Program..

[27]  Stephen J. Wright Superlinear Convergence of a Stabilized SQP Method to a Degenerate Solution , 1998, Comput. Optim. Appl..

[28]  J. Gondzio,et al.  Regularized Symmetric Indefinite Systems in Interior Point Methods for Linear and Quadratic Optimization , 1999 .

[29]  Roger Fletcher,et al.  Stable reduced Hessian updates for indefinite quadratic programming , 2000, Math. Program..

[30]  P. Toint,et al.  An iterative working-set method for large-scale nonconvex quadratic programming , 2002 .

[31]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[32]  P. Toint,et al.  Numerical Methods for Large-Scale Non-Convex Quadratic Programming , 2002 .

[33]  A. Forsgren Inertia-controlling factorizations for optimization algorithms , 2002 .

[34]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[35]  Jean Charles Gilbert,et al.  Global linear convergence of an augmented Lagrangian algorithm for solving convex quadratic optimization problems , 2002 .

[36]  José Luis Morales,et al.  A numerical study of limited memory BFGS methods , 2002, Appl. Math. Lett..

[37]  Nicholas I. M. Gould,et al.  GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization , 2003, TOMS.

[38]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[39]  Iain S. Duff,et al.  MA57---a code for the solution of sparse symmetric definite and indefinite systems , 2004, TOMS.

[40]  K. I. M. McKinnon,et al.  The simplest examples where the simplex method cycles and conditions where expand fails to prevent cycling , 2000, Math. Program..

[41]  L. Biegler,et al.  QPSchur: A dual, active-set, Schur-complement method for large-scale and structured convex quadratic programming , 2006 .

[42]  N. Gould,et al.  Finding a point in the relative interior of a polyhedron , 2007 .

[43]  Hans Joachim Ferreau,et al.  An online active set strategy to overcome the limitations of explicit MPC , 2008 .

[44]  H. Huynh A Large-Scale Quadratic Programming Solver Based on Block-Lu Updates of the KKT System , 2008 .

[45]  H. Bock,et al.  RELIABLE SOLUTION OF CONVEX QUADRATIC PROGRAMS WITH PARAMETRIC ACTIVE SET METHODS , 2010 .

[46]  A.,et al.  How the augmented Lagrangian algorithm can deal with an infeasible convex quadratic optimization problem , 2010 .

[47]  Elizabeth Wong Active-Set Methods for Quadratic Programming , 2011 .

[48]  lexander,et al.  THE GENERALIZED SIMPLEX METHOD FOR MINIMIZING A LINEAR FORM UNDER LINEAR INEQUALITY RESTRAINTS , 2012 .

[49]  Daniel P. Robinson,et al.  A Globally Convergent Stabilized SQP Method , 2013, SIAM J. Optim..

[50]  Jean Charles Gilbert,et al.  OQLA/QPALM – Convex quadratic optimization solvers using the augmented Lagrangian approach, with an appropriate behavior on infeasible or unbounded problems , 2014 .

[51]  Christian Kirches,et al.  qpOASES: a parametric active-set algorithm for quadratic programming , 2014, Mathematical Programming Computation.

[52]  Nicholas I. M. Gould,et al.  CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization , 2013, Computational Optimization and Applications.

[53]  Stephen J. Wright,et al.  Quadratic programming , 2015 .

[54]  Philip E. Gill,et al.  Methods for convex and general quadratic programming , 2014, Mathematical Programming Computation.

[55]  Daniel P. Robinson,et al.  A globally convergent primal-dual active-set framework for large-scale convex quadratic optimization , 2015, Comput. Optim. Appl..

[56]  Nicholas I. M. Gould,et al.  Adaptive augmented Lagrangian methods: algorithms and practical numerical experience , 2014, Optim. Methods Softw..

[57]  Michael A. Saunders,et al.  User's Guide for SQOPT Version 7.5: Software for Large-Scale Linear and Quadratic Programming , 2016 .