SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization

Sequential quadratic programming (SQP) methods have proved highly effective for solving constrained optimization problems with smooth nonlinear functions in the objective and constraints. Here we consider problems with general inequality constraints (linear and nonlinear). We assume that first derivatives are available and that the constraint gradients are sparse. We discuss an SQP algorithm that uses a smooth augmented Lagrangian merit function and makes explicit provision for infeasibility in the original problem and the QP subproblems. SNOPT is a particular implementation that makes use of a semidefinite QP solver. It is based on a limited-memory quasi-Newton approximation to the Hessian of the Lagrangian and uses a reduced-Hessian algorithm (SQOPT) for solving the QP subproblems. It is designed for problems with many thousands of constraints and variables but a moderate number of degrees of freedom (say, up to 2000). An important application is to trajectory optimization in the aerospace industry. Numerical results are given for most problems in the CUTE and COPS test collections (about 900 examples).

[1]  George B. Dantzig,et al.  Linear programming and extensions , 1965 .

[2]  R. Bartels.  A stabilization of the simplex method , 1971 .

[3]  S. M. Robinson,et al.  A quadratically-convergent algorithm for general nonlinear programming problems , 1972, Math. Program..

[4]  A. Conn.  Constrained Optimization Using a Nondifferentiable Penalty Function , 1973 .

[5]  A. Gourlay,et al.  Rank-one and Rank-two Corrections to Positive Definite Matrices Expressed in Product Form , 1973 .

[6]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[7]  R. Tapia.  A stable approach to Newton's method for general mathematical programming problems inRn , 1975 .

[8]  D. Goldfarb.  Factorized variable metric methods for unconstrained optimization , 1976 .

[9]  Shih-Ping Han,et al.  Superlinearly convergent variable metric algorithms for general nonlinear programming problems , 1976, Math. Program..

[10]  A. Conn.  Linear Programming via a Nondifferentiable Penalty Function , 1976 .

[11]  M. J. D. Powell,et al.  A fast algorithm for nonlinearly constrained optimization calculations , 1978 .

[12]  M. J. D. Powell,et al.  Algorithms for nonlinear constraints that use lagrangian functions , 1978, Math. Program..

[13]  M. J. D. Powell,et al.  THE CONVERGENCE OF VARIABLE METRIC METHODS FOR NONLINEARLY CONSTRAINED OPTIMIZATION CALCULATIONS , 1978 .

[14]  Michael A. Saunders,et al.  Large-scale linearly constrained optimization , 1978, Math. Program..

[15]  Philip E. Gill,et al.  The computation of Lagrange-multiplier estimates for constrained minimization , 1979, Math. Program..

[16]  Klaus Schittkowski,et al.  Test examples for nonlinear programming codes , 1980 .

[17]  R. Bartels.  A penalty linear programming method using reduced-gradient basis-exchange techniques☆ , 1980 .

[18]  Iain S. Duff,et al.  MA28 --- A set of Fortran subroutines for sparse unsymmetric linear equations , 1980 .

[19]  R. Schnabel,et al.  A NEW DERIVATION OF SYMMETRIC POSITIVE DEFINITE SECANT UPDATES , 1980 .

[20]  P. Spellucci.  Han's method without solving QP , 1981 .

[21]  Philip E. Gill,et al.  Practical optimization , 1981 .

[22]  G. Hoek.  Asymptotic properties of reduction methods applying linearly equality constrained reduced problems , 1982 .

[23]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[24]  P. Boggs,et al.  On the Local Convergence of Quasi-Newton Methods for Constrained Optimization , 1982 .

[25]  R. Dembo,et al.  INEXACT NEWTON METHODS , 1982 .

[26]  Michael A. Saunders,et al.  A projected Lagrangian algorithm and its implementation for sparse nonlinear constraints , 1982 .

[27]  Robert Fourer,et al.  Solving staircase linear programs by the simplex method, 1: Inversion , 1982, Math. Program..

[28]  John K. Reid,et al.  A sparsity-exploiting variant of the Bartels—Golub decomposition for linear programming bases , 1982, Math. Program..

[29]  M. J. D. Powell,et al.  Variable Metric Methods for Constrained Optimization , 1982, ISMP.

[30]  Kaoru Tone,et al.  Revisions of constraint approximations in the successive QP method for nonlinear programming problems , 1983, Math. Program..

[31]  Robert Fourer,et al.  Solving staircase linear programs by the simplex method, 2: Pricing , 1983, Math. Program..

[32]  Albert G. Buckley,et al.  QN-like variable storage conjugate gradients , 1983, Math. Program..

[33]  Michael A. Saunders,et al.  User''s guide for NPSOL (Ver-sion 4.0): A FORTRAN package for nonlinear programming , 1984 .

[34]  P. Gill,et al.  Sparse Matrix Methods in Optimization , 1984 .

[35]  Arne Drud,et al.  CONOPT: A GRG code for large sparse dynamic nonlinear optimization problems , 1985, Math. Program..

[36]  Robert Fourer,et al.  A simplex algorithm for piecewise-linear programming I: Derivation and proof , 1985, Math. Program..

[37]  Albert G. Buckley,et al.  Algorithm 630: BBVSCG–a variable-storage algorithm for function minimization , 1985, TOMS.

[38]  P. Gill,et al.  Some theoretical properties of an augmented lagrangian merit function , 1986 .

[39]  Klaus Schittkowski,et al.  NLPQL: A fortran subroutine solving constrained nonlinear programming problems , 1986 .

[40]  Roger Fletcher,et al.  Practical methods of optimization; (2nd ed.) , 1987 .

[41]  C. Hargraves,et al.  DIRECT TRAJECTORY OPTIMIZATION USING NONLINEAR PROGRAMMING AND COLLOCATION , 1987 .

[42]  P. Gill,et al.  Maintaining LU factors of a general sparse matrix , 1987 .

[43]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[44]  Robert Fourer,et al.  A simplex algorithm for piecewise-linear programming II: Finiteness, feasibility and degeneracy , 1988, Math. Program..

[45]  Nicholas J. Higham,et al.  FORTRAN codes for estimating the one-norm of a real or complex matrix, with applications to condition estimation , 1988, TOMS.

[46]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[47]  Michael A. Saunders,et al.  A practical anti-cycling procedure for linearly constrained optimization , 1989, Math. Program..

[48]  Claude Lemaréchal,et al.  Some numerical experiments with variable-storage quasi-Newton algorithms , 1989, Math. Program..

[49]  S. I. Feldman,et al.  A Fortran to C converter , 1990, FORF.

[50]  Shirley Dex,et al.  JR 旅客販売総合システム（マルス）における運用及び管理について , 1991 .

[51]  Michael A. Saunders,et al.  Inertia-Controlling Methods for General Quadratic Programming , 1991, SIAM Rev..

[52]  Jorge Nocedal,et al.  An analysis of reduced Hessian methods for constrained optimization , 1991, Math. Program..

[53]  L. Biegler,et al.  Simultaneous solution and optimization strategies for parameter estimation of differential-algebraic equation systems , 1991 .

[54]  N. Gould.  An Algorithm for Large-Scale Quadratic Programming , 1991 .

[55]  S. K. Eldersveld.  Large-scale sequential quadratic programming algorithms , 1992 .

[56]  P. Toint,et al.  Lancelot: A FORTRAN Package for Large-Scale Nonlinear Optimization (Release A) , 1992 .

[57]  Robert Fourer,et al.  A simplex algorithm for piecewise-linear programming III: Computational analysis and applications , 1992, Math. Program..

[58]  Brian W. Kernighan,et al.  AMPL: A Modeling Language for Mathematical Programming , 1993 .

[59]  J. Betts,et al.  A sparse nonlinear optimization algorithm , 1994 .

[60]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[61]  Francisco J. Prieto,et al.  A Sequential Quadratic Programming Algorithm Using an Incomplete Solution of the Subproblem , 1995, SIAM J. Optim..

[62]  Jorge Nocedal,et al.  A Reduced Hessian Method for Large-Scale Constrained Optimization , 1995, SIAM J. Optim..

[63]  Paul T. Boggs,et al.  Sequential Quadratic Programming , 1995, Acta Numerica.

[64]  Jorge Nocedal,et al.  A Limited Memory Algorithm for Bound Constrained Optimization , 1995, SIAM J. Sci. Comput..

[65]  Leon S. Lasdon,et al.  Intopt: an interior point algorithm for large scale nonlinear optimization , 1995 .

[66]  Nicholas I. M. Gould,et al.  CUTE: constrained and unconstrained testing environment , 1995, TOMS.

[67]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[68]  Walter Murray,et al.  Sequential Quadratic Programming Methods for Large-Scale Problems , 1997, Comput. Optim. Appl..

[69]  Nicholas I. M. Gould,et al.  A numerical comparison between the LANCELOT and MINOS packages for large-scale constrained optimization , 1997 .

[70]  Jorge Nocedal,et al.  On the Implementation of an Algorithm for Large-Scale Equality Constrained Optimization , 1998, SIAM J. Optim..

[71]  Sven Leyffer,et al.  User manual for filterSQP , 1998 .

[72]  Peter Spellucci,et al.  A new technique for inconsistent QP problems in the SQP method , 1998, Math. Methods Oper. Res..

[73]  Todd Plantenga,et al.  A Trust Region Method for Nonlinear Programming Based on Primal Interior-Point Techniques , 1998, SIAM J. Sci. Comput..

[74]  Peter Spellucci,et al.  An SQP method for general nonlinear programs using only equality constrained subproblems , 1998, Math. Program..

[75]  Robert J. Vanderbei,et al.  An Interior-Point Algorithm for Nonconvex Nonlinear Programming , 1999, Comput. Optim. Appl..

[76]  Nicholas I. M. Gould,et al.  SQP Methods for Large-Scale Nonlinear Programming , 1999, System Modelling and Optimization.

[77]  Jorge Nocedal,et al.  An Interior Point Algorithm for Large-Scale Nonlinear Programming , 1999, SIAM J. Optim..

[78]  Paul T. Boggs,et al.  A Global Convergence Analysis of an Algorithm for Large-Scale Nonlinear Optimization Problems , 1999, SIAM J. Optim..

[79]  Paul T. Boggs,et al.  A Practical Algorithm for General Large Scale Nonlinear Optimization Problems , 1999, SIAM J. Optim..

[80]  Jorge J. Moré,et al.  COPS: Large-scale nonlinearly constrained optimization problems , 2000 .

[81]  Jorge Nocedal,et al.  A trust region method based on interior point techniques for nonlinear programming , 2000, Math. Program..

[82]  Jorge Nocedal,et al.  Automatic Preconditioning by Limited Memory Quasi-Newton Updating , 1999, SIAM J. Optim..

[83]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[84]  R.W.H. Sargent,et al.  A New SQP Algorithm for Large-Scale Nonlinear Programming , 2000, SIAM J. Optim..

[85]  Todd Munson,et al.  Benchmarking optimization software with COPS. , 2001 .

[86]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[87]  Jorge J. Moré,et al.  Benchmarking optimization software with COPS. , 2001 .

[88]  Sven Leyffer,et al.  Nonlinear programming without a penalty function , 2002, Math. Program..

[89]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[90]  José Luis Morales,et al.  A numerical study of limited memory BFGS methods , 2002, Appl. Math. Lett..

[91]  A. F. Veinott,et al.  New methods for dynamic programming over an infinite time horizon , 2002 .

[92]  Michael A. Saunders,et al.  USER’S GUIDE FOR SNOPT 5.3: A FORTRAN PACKAGE FOR LARGE-SCALE NONLINEAR PROGRAMMING , 2002 .

[93]  Philip E. Gill,et al.  Limited-Memory Reduced-Hessian Methods for Large-Scale Unconstrained Optimization , 2003, SIAM J. Optim..

[94]  Nicholas I. M. Gould,et al.  GALAHAD, a library of thread-safe Fortran 90 packages for large-scale nonlinear optimization , 2003, TOMS.

[95]  Nicholas I. M. Gould,et al.  CUTEr and SifDec: A constrained and unconstrained testing environment, revisited , 2003, TOMS.

[96]  Todd Munson,et al.  Benchmarking optimization software with COPS 3.0. , 2001 .