A direct active set algorithm for large sparse quadratic programs with simple bounds

We show how a direct active set method for solving definite and indefinite quadratic programs with simple bounds can be efficiently implemented for large sparse problems. All of the necessary factorizations can be carried out in a static data structure that is set up before the numeric computation begins. The space required for these factorizations is no larger than that required for a single sparse Cholesky factorization of the Hessian of the quadratic. We propose several improvements to this basic algorithm: a new way to find a search direction in the indefinite case that allows us to free more than one variable at a time and a new heuristic method for finding a starting point. These ideas are motivated by the two-norm trust region problem. Additionally, we also show how projection techniques can be used to add several constraints to the active set at each iteration. Our experimental results show that an algorithm with these improvements runs much faster than the basic algorithm for positive definite problems and finds local minima with lower function values for indefinite problems.

[1]  S. Parter.  The Use of Linear Graphs in Gauss Elimination , 1961 .

[2]  R. Fletcher.  A General Quadratic Programming Algorithm , 1971 .

[3]  R. Fletcher,et al.  Minimization of a Quadratic Function of Many Variables Subject only to Lower and Upper Bounds , 1974 .

[4]  Robert E. Tarjan,et al.  Algorithmic Aspects of Vertex Elimination on Graphs , 1976, SIAM J. Comput..

[5]  Philip E. Gill,et al.  Numerically stable methods for quadratic programming , 1978, Math. Program..

[6]  Gordon C. Everstine,et al.  A comparasion of three resequencing algorithms for the reduction of matrix profile and wavefront , 1979 .

[7]  D. Bertsekas.  Projected Newton methods for optimization problems with simple constraints , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[8]  M. Yannakakis.  Computing the Minimum Fill-in is NP^Complete , 1981 .

[9]  Alan George,et al.  Computer Solution of Large Sparse Positive Definite , 1981 .

[10]  Philip E. Gill,et al.  Practical optimization , 1981 .

[11]  J. Pasciak,et al.  Computer solution of large sparse positive definite systems , 1982 .

[12]  Jorge J. Moré,et al.  Computing a Trust Region Step , 1983 .

[13]  Gene H. Golub,et al.  Matrix computations , 1983 .

[14]  J. Gilbert,et al.  Sparse Partial Pivoting in Time Proportional to Arithmetic Operations , 1986 .

[15]  Katta G. Murty,et al.  Some NP-complete problems in quadratic and nonlinear programming , 1987, Math. Program..

[16]  Jack Dongarra,et al.  LINPACK Users' Guide , 1987 .

[17]  Paul H. Calamai,et al.  Projected gradient methods for linearly constrained problems , 1987, Math. Program..

[18]  P. Toint,et al.  Testing a class of methods for solving minimization problems with simple bounds on the variables , 1988 .

[19]  P. Toint,et al.  Global convergence of a class of trust region algorithms for optimization with simple bounds , 1988 .

[20]  Å. Björck.  A direct method for sparse least squares problems with lower and upper bounds , 1988 .

[21]  Nicholas J. Higham,et al.  FORTRAN codes for estimating the one-norm of a real or complex matrix, with applications to condition estimation , 1988, TOMS.

[22]  Thomas F. Coleman,et al.  A chordal preconditioner for large-scale optimization , 1988, Math. Program..

[23]  J. J. Moré,et al.  Algorithms for bound constrained quadratic programming problems , 1989 .