An SR1/BFGS SQP algorithm for nonconvex nonlinear programs with block-diagonal Hessian matrix

We present a quasi-Newton sequential quadratic programming (SQP) algorithm for nonlinear programs in which the Hessian of the Lagrangian function is block-diagonal. Problems with this characteristic frequently arise in the context of optimal control; for example, when a direct multiple shooting parametrization is used. In this article, we describe an implementation of a filter line-search SQP method that computes search directions using an active-set quadratic programming (QP) solver. To take advantage of the block-diagonal structure of the Hessian matrix, each block is approximated separately by quasi-Newton updates. For nonconvex instances, that arise, for example, in optimum experimental design control problems, these blocks are often found to be indefinite. In that case, the block-BFGS quasi-Newton update can lead to poor convergence. The novel aspect in this work is the use of SR1 updates in place of BFGS approximations whenever possible. The resulting indefinite QPs necessitate an inertia control mechanism within the sparse Schur-complement factorization that is carried out by the active-set QP solver. This permits an adaptive selection of the Hessian approximation that guarantees sufficient progress towards a stationary point of the problem. Numerical results demonstrate that the proposed approach reduces the number of SQP iterations and CPU time required for the solution of a set of optimal control problems.

[1]  Dennis Janka,et al.  Sequential quadratic programming with indefinite Hessian approximations for nonlinear optimum experimental design for parameter estimation in differential–algebraic equations , 2015 .

[2]  Richard H. Byrd,et al.  Analysis of a Symmetric Rank-One Trust Region Method , 1996, SIAM J. Optim..

[3]  Hans Bock,et al.  Direct Multiple Shooting for Nonlinear Optimum Experimental Design , 2015 .

[4]  P. Toint,et al.  Testing a class of methods for solving minimization problems with simple bounds on the variables , 1988 .

[5]  M. J. D. Powell,et al.  THE CONVERGENCE OF VARIABLE METRIC METHODS FOR NONLINEARLY CONSTRAINED OPTIMIZATION CALCULATIONS , 1978 .

[6]  Johannes P. Schlöder,et al.  An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization: Part II: Software aspects and applications , 2003, Comput. Chem. Eng..

[7]  Christian Kirches,et al.  qpOASES: a parametric active-set algorithm for quadratic programming , 2014, Mathematical Programming Computation.

[8]  George M. Siouris,et al.  Applied Optimal Control: Optimization, Estimation, and Control , 1979, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[10]  Nicholas I. M. Gould,et al.  Lancelot: A FORTRAN Package for Large-Scale Nonlinear Optimization (Release A) , 1992 .

[11]  Nicholas I. M. Gould,et al.  Convergence of quasi-Newton matrices generated by the symmetric rank one update , 1991, Math. Program..

[12]  P. Gill,et al.  A Schur-complement method for sparse quadratic programming , 1987 .

[13]  Robert J. Vanderbei,et al.  An Interior-Point Algorithm for Nonconvex Nonlinear Programming , 1999, Comput. Optim. Appl..

[14]  Moritz Diehl,et al.  An approximation technique for robust nonlinear optimization , 2006, Math. Program..

[15]  Richard H. Byrd,et al.  A Theoretical and Experimental Study of the Symmetric Rank-One Update , 1993, SIAM J. Optim..

[16]  J. E. Cuthrell,et al.  Simultaneous optimization and solution methods for batch reactor control profiles , 1989 .

[17]  Iain S. Duff,et al.  MA57---a code for the solution of sparse symmetric definite and indefinite systems , 2004, TOMS.

[18]  P. Toint,et al.  Local convergence analysis for partitioned quasi-Newton updates , 1982 .

[19]  D. Luenberger,et al.  SELF-SCALING VARIABLE METRIC ( SSVM ) ALGORITHMS Part I : Criteria and Sufficient Conditions for Scaling a Class of Algorithms * t , 2007 .

[20]  H. J. Ferreau,et al.  An online active set strategy to overcome the limitations of explicit MPC , 2008 .

[21]  M. Diehl,et al.  Numerical Methods for Optimal Control with Binary Control Functions Applied to a Lotka-Volterra Type Fishing Problem , 2006 .

[22]  Lorenz T. Biegler,et al.  Line Search Filter Methods for Nonlinear Programming: Motivation and Global Convergence , 2005, SIAM J. Optim..

[23]  Johannes P. Schlöder,et al.  An efficient multiple shooting based reduced SQP strategy for large-scale dynamic process optimization. Part 1: theoretical aspects , 2003, Comput. Chem. Eng..

[24]  F. Pukelsheim.  Optimal Design of Experiments , 1993 .

[25]  Martin Hermann,et al.  Numerical Methods for Initial Value Problems , 2014 .

[26]  Todd Munson,et al.  Benchmarking optimization software with COPS 3.0. , 2001 .

[27]  J. Reid,et al.  An approximate minimum degree algorithm for matrices with dense rows , 2008 .

[28]  Jorge J. Moré,et al.  Benchmarking optimization software with performance profiles , 2001, Math. Program..

[29]  Roger Fletcher,et al.  Stable reduced Hessian updates for indefinite quadratic programming , 2000, Math. Program..

[30]  Sebastian Sager,et al.  Sampling Decisions in Optimum Experimental Design in the Light of Pontryagin's Maximum Principle , 2013, SIAM J. Control. Optim..

[31]  P. Toint,et al.  An iterative working-set method for large-scale nonconvex quadratic programming , 2002 .

[32]  Stefan Körkel,et al.  Numerische Methoden für Optimale Versuchsplanungsprobleme bei nichtlinearen DAE-Modellen , 2002 .

[33]  P. Toint,et al.  Partitioned variable metric updates for large structured optimization problems , 1982 .

[34]  Christian Bischof,et al.  Adifor 2.0: automatic differentiation of Fortran 77 programs , 1996 .

[35]  Alexander Meeraus,et al.  Matrix augmentation and partitioning in the updating of the basis inverse , 1977, Math. Program..

[36]  R. Tapia,et al.  Sizing the BFGS and DFP updates: Numerical study , 1993 .

[37]  Philip E. Gill,et al.  Methods for convex and general quadratic programming , 2014, Mathematical Programming Computation.

[38]  H. Bock,et al.  A Multiple Shooting Algorithm for Direct Solution of Optimal Control Problems , 1984 .

[39]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[40]  Fraser J Forbes,et al.  Model Structure and Adjustable Parameter Selection for Operations Optimization , 1994 .

[41]  Panos M. Pardalos,et al.  Quadratic programming with one negative eigenvalue is NP-hard , 1991, J. Glob. Optim..

[42]  D. Luenberger,et al.  Self-Scaling Variable Metric (SSVM) Algorithms , 1974 .

[43]  R. Sargent,et al.  Solution of a Class of Multistage Dynamic Optimization Problems. 2. Problems with Path Constraints , 1994 .