A Matrix-Free Trust-Region SQP Method for Equality Constrained Optimization

We develop and analyze a trust-region sequential quadratic programming (SQP) method for the solution of smooth equality constrained optimization problems, which allows the inexact and hence iterative solution of linear systems. Iterative solution of linear systems is important in large-scale applications, such as optimization problems with partial differential equation constraints, where direct solves are either too expensive or not applicable. Our trust-region SQP algorithm is based on a composite-step approach that decouples the step into a quasi-normal and a tangential step. The algorithm includes critical modifications of substep computations needed to cope with the inexact solution of linear systems. The global convergence of our algorithm is guaranteed under rather general conditions on the substeps. We propose algorithms to compute the substeps and prove that these algorithms satisfy global convergence conditions. All components of the resulting algorithm are specified in such a way that they can b...

[1]  Nicholas I. M. Gould,et al.  Trust Region Methods , 2000, MOS-SIAM Series on Optimization.

[2]  T. Steihaug.  The Conjugate Gradient Method and Trust Regions in Large Scale Optimization , 1983 .

[3]  Yvan Notay.  Flexible Conjugate Gradients , 2000, SIAM J. Sci. Comput..

[4]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[5]  Andreas Griewank,et al.  On constrained optimization by adjoint based quasi-Newton methods , 2002, Optim. Methods Softw..

[6]  Ekkehard W. Sachs,et al.  Global Convergence of Inexact Reduced SQP Methods , 1995, Universität Trier, Mathematik/Informatik, Forschungsbericht.

[7]  Andrea Walther.  A First-Order Convergence Analysis of Trust-Region Methods with Inexact Jacobians , 2008, SIAM J. Optim..

[8]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[9]  Nicholas I. M. Gould,et al.  On the Solution of Equality Constrained Quadratic Programming Problems Arising in Optimization , 2001, SIAM J. Sci. Comput..

[10]  Nicholas I. M. Gould,et al.  Preconditioning Saddle-Point Systems with Applications in Optimization , 2010, SIAM J. Sci. Comput..

[11]  L. N. Vicente,et al.  Trust-Region Interior-Point SQP Algorithms for a Class of Nonlinear Programming Problems , 1998 .

[12]  Valeria Simoncini,et al.  Theory of Inexact Krylov Subspace Methods and Applications to Scientific Computing , 2003, SIAM J. Sci. Comput..

[13]  Jorge Nocedal,et al.  An Interior Point Algorithm for Large-Scale Nonlinear Programming , 1999, SIAM J. Optim..

[14]  Tamara G. Kolda,et al.  An overview of the Trilinos project , 2005, TOMS.

[15]  Philippe L. Toint,et al.  Towards an efficient sparsity exploiting newton method for minimization , 1981 .

[16]  John E. Dennis,et al.  A Global Convergence Theory for General Trust-Region-Based Algorithms for Equality Constrained Optimization , 1997, SIAM J. Optim..

[17]  L. S. Hou,et al.  Numerical Approximation of Optimal Flow Control Problems by a Penalty Method: Error Estimates and Numerical Results , 1999, SIAM J. Sci. Comput..

[18]  Martin Burger,et al.  Fast Optimal Design of Semiconductor Devices , 2003, SIAM J. Appl. Math..

[19]  Matthias Heinkenschloss,et al.  Integration of Sequential Quadratic Programming and Domain Decomposition Methods for Nonlinear Optimal Control Problems , 2008 .

[20]  Gene H. Golub,et al.  Numerical solution of saddle point problems , 2005, Acta Numerica.

[21]  George Biros,et al.  Inexactness Issues in the Lagrange-Newton-Krylov-Schur Method for PDE-constrained Optimization , 2003 .

[22]  George Biros,et al.  Parallel Lagrange-Newton-Krylov-Schur Methods for PDE-Constrained Optimization. Part I: The Krylov-Schur Solver , 2005, SIAM J. Sci. Comput..

[23]  Luís N. Vicente,et al.  Analysis of Inexact Trust-Region SQP Algorithms , 2002, SIAM J. Optim..

[24]  Xiao-Chuan Cai,et al.  Parallel Full Space SQP Lagrange-Newton-Krylov-Schwarz Algorithms for PDE-Constrained Optimization Problems , 2005, SIAM J. Sci. Comput..

[25]  Max Gunzburger,et al.  Finite element approximations of an optimal control problem associated with the scalar Ginzburg-Landau equation , 1991 .

[26]  Hoang Nguyen,et al.  Neumann-Neumann Domain Decomposition Preconditioners for Linear-Quadratic Elliptic Optimal Control Problems , 2006, SIAM J. Sci. Comput..

[27]  Denis Ridzal,et al.  Trust Region SQP Methods With Inexact Linear System Solves For Large-Scale Optimization , 2006 .

[28]  Matthias Heinkenschloss,et al.  An Inexact Trust-Region SQP Method with Applications to PDE-Constrained Optimization , 2008 .

[29]  Stefan Ulbrich,et al.  Adaptive Multilevel Inexact SQP Methods for PDE-Constrained Optimization , 2011, SIAM J. Optim..

[30]  Gene H. Golub,et al.  Inexact Preconditioned Conjugate Gradient Method with Inner-Outer Iteration , 1999, SIAM J. Sci. Comput..

[31]  M. Heinkenschloss,et al.  Domain decomposition methods for advection dominated linear-quadratic elliptic optimal control problems , 2005 .

[32]  Jorge Nocedal,et al.  An inexact Newton method for nonconvex equality constrained optimization , 2009, Math. Program..

[33]  M. Heinkenschloss,et al.  Numerical study of a matrix-free trust-region SQP method for equality constrained optimization , 2011 .

[34]  A. Borzì,et al.  An efficient algebraic multigrid method for solving optimality systems , 2004 .

[35]  O. Axelsson,et al.  A black box generalized conjugate gradient solver with inner iterations and variable-step preconditioning , 1991 .

[36]  John E. Dennis,et al.  On the Convergence Theory of Trust-Region-Based Algorithms for Equality-Constrained Optimization , 1997, SIAM J. Optim..

[37]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[38]  Jorge Nocedal,et al.  An Inexact SQP Method for Equality Constrained Optimization , 2008, SIAM J. Optim..

[39]  Alfio Borzì,et al.  Multigrid Methods for PDE Optimization , 2009, SIAM Rev..