Numerical Methods for Large-Scale Non-Convex Quadratic Programming

We consider numerical methods for finding (weak) second-order critical points for large-scale non-convex quadratic programming problems. We describe two new methods. The first is of the active-set variety. Although convergent from any starting point, it is intended primarily for the case where a good estimate of the optimal active set can be predicted. The second is an interior-point trust-region type, and has proved capable of solving problems involving up to half a million unknowns and constraints. The solution of a key equality constrained subproblem, common to both methods, is described. The results of comparative tests on a large set of convex and non-convex quadratic programming examples are given.

[1]  K. Swarup.  Indefinite Quadratic Programming with Non-linear Constraints , 1967 .

[2]  Anthony V. Fiacco,et al.  Nonlinear programming;: Sequential unconstrained minimization techniques , 1968 .

[3]  Boris Polyak.  The conjugate gradient method in extremal problems , 1969 .

[4]  J. Bunch,et al.  Direct Methods for Solving Symmetric Indefinite Systems of Linear Equations , 1971 .

[5]  R. Fletcher.  A General Quadratic Programming Algorithm , 1971 .

[6]  J. Bunch,et al.  Some stable methods for calculating inertia and solving symmetric linear systems , 1977 .

[7]  R. Fletcher.  Factorizing symmetric indefinite matrices , 1976 .

[8]  Alexander Meeraus,et al.  Matrix augmentation and partitioning in the updating of the basis inverse , 1977, Math. Program..

[9]  D. Sorensen.  Updating the Symmetric Indefinite Factorization with Applications in a Modified Newton's Method , 1977 .

[10]  David A. H. Jacobs,et al.  The State of the Art in Numerical Analysis. , 1978 .

[11]  Philip E. Gill,et al.  Numerically stable methods for quadratic programming , 1978, Math. Program..

[12]  Alfred Auslender,et al.  Penalty methods for computing points that satisfy second order necessary conditions , 1979, Math. Program..

[13]  I. Duff,et al.  Direct Solution of Sets of Linear Equations whose Matrix is Sparse, Symmetric and Indefinite , 1979 .

[14]  C. B. Luis.  Une caractérisation complète des minima locaux en programmation quadratique , 1980 .

[15]  O. L. Mangasarian,et al.  Locally unique solutions of quadratic programs, linear and nonlinear complementarity problems , 1980, Math. Program..

[16]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[17]  Shih-Ping Han.  SOLVING QUADRATIC PROGRAMS BY AN EXACT PENALTY FUNCTION , 1981 .

[18]  J. Borwein.  Necessary and sufficient conditions for quadratic minimality , 1982 .

[19]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[20]  John K. Reid,et al.  The Multifrontal Solution of Indefinite Sparse Symmetric Linear , 1983, TOMS.

[21]  I. Duff.  Sparse Matrices and Their Uses. , 1983 .

[22]  Gene H. Golub,et al.  Matrix computations , 1983 .

[23]  N. Gould,et al.  On the Location of Directions of Infinite Descent for Nonlinear Programming Algorithms , 1984 .

[24]  J. Crouzeix,et al.  Definiteness and semidefiniteness of quadratic forms revisited , 1984 .

[25]  Nicholas I. M. Gould,et al.  On practical conditions for the existence and uniqueness of solutions to the general equality quadratic programming problem , 1985, Math. Program..

[26]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[27]  P. Gill,et al.  A Schur-complement method for sparse quadratic programming , 1987 .

[28]  P. Toint,et al.  Global convergence of a class of trust region algorithms for optimization with simple bounds , 1988 .

[29]  P. Pardalos,et al.  Checking local optimality in constrained quadratic programming is NP-hard , 1988 .

[30]  Josef Stoer,et al.  Numerische Mathematik 1 , 1989 .

[31]  Stephen A. Vavasis,et al.  Quadratic Programming is in NP , 1990, Inf. Process. Lett..

[32]  Michael A. Saunders,et al.  Inertia-Controlling Methods for General Quadratic Programming , 1991, SIAM Rev..

[33]  I. Duff,et al.  The factorization of sparse symmetric indefinite matrices , 1991 .

[34]  S. Vavasis.  Nonlinear optimization: complexity issues , 1991 .

[35]  N. Gould.  An Algorithm for Large-Scale Quadratic Programming , 1991 .

[36]  John M. Mulvey,et al.  Higher-Order Predictor-Corrector Interior Point Methods with Application to Quadratic Objectives , 1993, SIAM J. Optim..

[37]  Robert J. Vanderbei,et al.  Symmetric indefinite systems for interior point methods , 1993, Math. Program..

[38]  John G. Lewis,et al.  Proceedings of the Fifth SIAM Conference on Applied Linear Algebra , 1994 .

[39]  Yin Zhang,et al.  On the Convergence of a Class of Infeasible Interior-Point Methods for the Horizontal Linear Complementarity Problem , 1994, SIAM J. Optim..

[40]  J. Gondzio,et al.  Presolove Analysis of Linear Programs Prior to Applying an Interior Point Method , 1997, INFORMS J. Comput..

[41]  Paul T. Boggs,et al.  Sequential Quadratic Programming , 1995, Acta Numerica.

[42]  Erling D. Andersen,et al.  Presolving in linear programming , 1995, Math. Program..

[43]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[44]  Jacek Gondzio,et al.  Implementation of Interior Point Methods for Large Scale Linear Programming , 1996 .

[45]  Stephen J. Wright,et al.  A superquadratic infeasible-interior-point method for linear complementarity problems , 1994, Math. Program..

[46]  K. M. Anstreicher,et al.  Interior point methods in mathematical programming , 1996 .

[47]  Jacek Gondzio,et al.  Presolove Analysis of Linear Programs Prior to Applying an Interior Point Method , 1997, INFORMS J. Comput..

[48]  Jacek Gondzio,et al.  Warm start of the primal-dual method applied in the cutting-plane scheme , 1998, Math. Program..

[49]  Yinyu Ye,et al.  Interior point algorithms: theory and analysis , 1997 .

[50]  J. Gondzio,et al.  Regularized Symmetric Indefinite Systems in Interior Point Methods for Linear and Quadratic Optimization , 1999 .

[51]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[52]  Nicholas I. M. Gould,et al.  A note on the convergence of barrier algorithms to second-order necessary points , 1999, Math. Program..

[53]  Nicholas I. M. Gould,et al.  SQP Methods for Large-Scale Nonlinear Programming , 1999, System Modelling and Optimization.

[54]  Jorge Nocedal,et al.  An Interior Point Algorithm for Large-Scale Nonlinear Programming , 1999, SIAM J. Optim..

[55]  Nicholas I. M. Gould,et al.  Solving the Trust-Region Subproblem using the Lanczos Method , 1999, SIAM J. Optim..

[56]  R. Vanderbei.  LOQO:an interior point code for quadratic programming , 1999 .

[57]  Jorge Nocedal,et al.  A trust region method based on interior point techniques for nonlinear programming , 2000, Math. Program..

[58]  Nicholas I. M. Gould,et al.  Constraint Preconditioning for Indefinite Linear Systems , 2000, SIAM J. Matrix Anal. Appl..

[59]  Nicholas I. M. Gould,et al.  A primal-dual trust-region algorithm for non-convex nonlinear programming , 2000, Math. Program..

[60]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[61]  Nicholas I. M. Gould,et al.  On the Solution of Equality Constrained Quadratic Programming Problems Arising in Optimization , 2001, SIAM J. Sci. Comput..

[62]  Thomas F. Coleman,et al.  A Preconditioned Conjugate Gradient Approach to Linear Equality Constrained Minimization , 2001, Comput. Optim. Appl..

[63]  Nicholas I. M. Gould,et al.  Superlinear Convergence of Primal-Dual Interior Point Algorithms for Nonlinear Programming , 2000, SIAM J. Optim..

[64]  Stephen J. Wright,et al.  Warm-Start Strategies in Interior-Point Methods for Linear Programming , 2002, SIAM J. Optim..

[65]  Nicholas I. M. Gould,et al.  Preprocessing for quadratic programming , 2004, Math. Program..

[66]  G. M. Ostrovsky,et al.  Linearly constrained optimization , 1989, Computing.

[67]  Gene H. Golub,et al.  Methods for modifying matrix factorizations , 1972, Milestones in Matrix Computation.