Methods for convex and general quadratic programming

Computational methods are considered for finding a point that satisfies the second-order necessary conditions for a general (possibly nonconvex) quadratic program (QP). The first part of the paper considers the formulation and analysis of an active-set method for a generic QP with both equality and inequality constraints. The method uses a search direction that is the solution of an equality-constrained subproblem involving a “working set” of linearly independent constraints. The method is a reformulation of a method for general QP first proposed by Fletcher, and modified subsequently by Gould. The reformulation facilitates a simpler analysis and has the benefit that the algorithm reduces to a variant of the simplex method when the QP is a linear program. The search direction is computed from a KKT system formed from the QP Hessian and the gradients of the working-set constraints. It is shown that, under certain circumstances, the solution of this KKT system may be updated using a simple recurrence relation, thereby giving a significant reduction in the number of KKT systems that need to be solved. The second part of the paper focuses on the solution of QP problems with constraints in so-called standard form. We describe how the constituent KKT systems are solved, and discuss how an initial basis is defined. Numerical results are presented for all QPs in the CUTEst test collection.

[1]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[2]  N. S. Barnett,et al.  Private communication , 1969 .

[3]  Richard W. Cottle,et al.  On classes of copositive matrices , 1970 .

[4]  R. Fletcher A General Quadratic Programming Algorithm , 1971 .

[5]  Antal Majthay Optimality conditions for quadratic programming , 1971, Math. Program..

[6]  John A. Tomlin,et al.  On pricing and backward transformation in linear programming , 1974, Math. Program..

[7]  Alexander Meeraus,et al.  Matrix augmentation and partitioning in the updating of the basis inverse , 1977, Math. Program..

[8]  Philip E. Gill,et al.  Numerically stable methods for quadratic programming , 1978, Math. Program..

[9]  C. B. Luis Une caractérisation complète des minima locaux en programmation quadratique , 1980 .

[10]  J. Bunch,et al.  A computational method for the indefinite quadratic programming problem , 1980 .

[11]  J. Borwein Necessary and sufficient conditions for quadratic minimality , 1982 .

[12]  John K. Reid,et al.  The Multifrontal Solution of Indefinite Sparse Symmetric Linear , 1983, TOMS.

[13]  Donald Goldfarb,et al.  A numerically stable dual method for solving strictly convex quadratic programs , 1983, Math. Program..

[14]  P. Gill,et al.  Sparse Matrix Methods in Optimization , 1984 .

[15]  Nicholas I. M. Gould,et al.  A weighted gram-schmidt method for convex quadratic programming , 1984, Math. Program..

[16]  Nicholas I. M. Gould,et al.  On practical conditions for the existence and uniqueness of solutions to the general equality quadratic programming problem , 1985, Math. Program..

[17]  M. Powell On the quadratic programming algorithm of Goldfarb and Idnani , 1985 .

[18]  Willard Miller,et al.  The IMA volumes in mathematics and its applications , 1986 .

[19]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[20]  P. Gill,et al.  A Schur-complement method for sparse quadratic programming , 1987 .

[21]  P. Gill,et al.  Maintaining LU factors of a general sparse matrix , 1987 .

[22]  P. Pardalos,et al.  Checking local optimality in constrained quadratic programming is NP-hard , 1988 .

[23]  Michael A. Saunders,et al.  A practical anti-cycling procedure for linearly constrained optimization , 1989, Math. Program..

[24]  P. Gill,et al.  On the identification of local minimizers in inertia-controlling methods for quadratic programming , 1991 .

[25]  Philip E. Gill,et al.  Numerical Linear Algebra and Optimization , 1991 .

[26]  Michael A. Saunders,et al.  Inertia-Controlling Methods for General Quadratic Programming , 1991, SIAM Rev..

[27]  Panos M. Pardalos,et al.  Quadratic programming with one negative eigenvalue is NP-hard , 1991, J. Glob. Optim..

[28]  Timothy A. Davis,et al.  An Unsymmetric-pattern Multifrontal Method for Sparse Lu Factorization , 1993 .

[29]  N. Gould An Algorithm for Large-Scale Quadratic Programming , 1991 .

[30]  J. H. Wilkinson,et al.  Reliable Numerical Computation. , 1992 .

[31]  Gautam Appa,et al.  Numerical Linear Algebra and Optimization: Volume 1 , 1992 .

[32]  Michael A. Saunders,et al.  A Block-LU Update for Large-Scale Linear Programming , 1992, SIAM J. Matrix Anal. Appl..

[33]  M. Best An Algorithm for the Solution of the Parametric Quadratic Programming Problem , 1996 .

[34]  Natashia Boland,et al.  A dual-active-set algorithm for positive semi-definite quadratic programming , 1997, Math. Program..

[35]  Cleve Ashcraft,et al.  SPOOLES: An Object-Oriented Sparse Matrix Library , 1999, PPSC.

[36]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[37]  Timothy A. Davis,et al.  A combined unifrontal/multifrontal method for unsymmetric sparse matrices , 1999, TOMS.

[38]  Patrick Amestoy,et al.  A Fully Asynchronous Multifrontal Solver Using Distributed Dynamic Scheduling , 2001, SIAM J. Matrix Anal. Appl..

[39]  P. Toint,et al.  An iterative working-set method for large-scale nonconvex quadratic programming , 2002 .

[40]  D. Ruiz A Scaling Algorithm to Equilibrate Both Rows and Columns Norms in Matrices 1 , 2001 .

[41]  Todd Munson,et al.  Benchmarking optimization software with COPS. , 2001 .

[42]  P. Toint,et al.  Numerical Methods for Large-Scale Non-Convex Quadratic Programming , 2002 .

[43]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[44]  Nicholas I. M. Gould,et al.  GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization , 2003, TOMS.

[45]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[46]  Timothy A. Davis,et al.  Algorithm 832: UMFPACK V4.3---an unsymmetric-pattern multifrontal method , 2004, TOMS.

[47]  Timothy A. Davis,et al.  A column pre-ordering strategy for the unsymmetric-pattern multifrontal method , 2004, TOMS.

[48]  Olaf Schenk,et al.  Solving unsymmetric sparse systems of linear equations with PARDISO , 2004, Future Gener. Comput. Syst..

[49]  Iain S. Duff,et al.  MA57---a code for the solution of sparse symmetric definite and indefinite systems , 2004, TOMS.

[50]  K. I. M. McKinnon,et al.  The simplest examples where the simplex method cycles and conditions where expand fails to prevent cycling , 2000, Math. Program..

[51]  L. Biegler,et al.  QPSchur: A dual, active-set, Schur-complement method for large-scale and structured convex quadratic programming , 2006 .

[52]  Sven Leyffer,et al.  Global and Finite Termination of a Two-Phase Augmented Lagrangian Filter Method for General Quadratic Programs , 2008, SIAM J. Sci. Comput..

[53]  Hans Joachim Ferreau,et al.  An online active set strategy to overcome the limitations of explicit MPC , 2008 .

[54]  H. Huynh A Large-Scale Quadratic Programming Solver Based on Block-Lu Updates of the KKT System , 2008 .

[55]  Elizabeth Wong Active-Set Methods for Quadratic Programming , 2011 .

[56]  Jennifer A. Scott,et al.  HSL_MA97 : a bit-compatible multifrontal code for sparse symmetric systems , 2011 .

[57]  W. Dorn Duality in Quadratic Programming... , 2011 .

[58]  Sven Leyffer,et al.  Mixed Integer Nonlinear Programming , 2011 .

[59]  Stefan Schäffler,et al.  Applied Mathematics and Parallel Computing: Festschrift for Klaus Ritter , 2012 .

[60]  P. Gill,et al.  Sequential Quadratic Programming Methods , 2012 .

[61]  Philippe L. Toint,et al.  CUTEst : a constrained testing environment with safe threads , 2013 .

[62]  Nicholas I. M. Gould,et al.  CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization , 2013, Computational Optimization and Applications.

[63]  Michael A. Saunders,et al.  User's Guide for SQOPT Version 7.5: Software for Large-Scale Linear and Quadratic Programming , 2016 .