A Regularized Factorization-Free Method for Equality-Constrained Optimization

We propose a method for equality-constrained optimization based on a problem in which all constraints are systematically regularized. The regularization is equivalent to applying an augmented Lagrangian method but the linear system used to compute a search direction is reminiscent of regularized sequential quadratic programming. A limited-memory BFGS approximation to second derivatives allows us to employ iterative methods for linear least squares to compute steps, resulting in a factorization-free implementation. We establish global and fast local convergence under weak assumptions. In particular, we do not require the LICQ and our method is suitable for degenerate problems. Preliminary numerical experiments show that a factorization-based implementation of our method exhibits significant robustness while a factorization-free implementation, though not as robust, is promising. We briefly discuss generalizing our framework to other classes of methods and to problems with inequality constraints.

[1]  Stephen J. Wright.  Superlinear Convergence of a Stabilized SQP Method to a Degenerate Solution , 1998, Comput. Optim. Appl..

[2]  Alexey F. Izmailov,et al.  Stabilized SQP revisited , 2012, Math. Program..

[3]  Daniel P. Robinson,et al.  A stabilized SQP method: global convergence , 2017 .

[4]  Iain S. Duff,et al.  MA57---a code for the solution of sparse symmetric definite and indefinite systems , 2004, TOMS.

[5]  Jorge Nocedal,et al.  On the limited memory BFGS method for large scale optimization , 1989, Math. Program..

[6]  Nicholas I. M. Gould,et al.  On practical conditions for the existence and uniqueness of solutions to the general equality quadratic programming problem , 1985, Math. Program..

[7]  Lorenz T. Biegler,et al.  On the implementation of an interior-point filter line-search algorithm for large-scale nonlinear programming , 2006, Math. Program..

[8]  Jacques Gauvin,et al.  A necessary and sufficient regularity condition to have bounded multipliers in nonconvex programming , 1977, Math. Program..

[9]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[10]  Mehiddin Al-Baali,et al.  Damped techniques for enforcing convergence of quasi-Newton methods , 2014, Optim. Methods Softw..

[11]  Alexey F. Izmailov,et al.  Combining stabilized SQP with the augmented Lagrangian algorithm , 2015, Comput. Optim. Appl..

[12]  Vincent Pateloup,et al.  Study of a primal-dual algorithm for equality constrained minimization , 2011, Comput. Optim. Appl..

[13]  M. Saunders,et al.  Solution of Sparse Indefinite Systems of Linear Equations , 1975 .

[14]  Jorge Nocedal,et al.  An inexact Newton method for nonconvex equality constrained optimization , 2009, Math. Program..

[15]  Daniel P. Robinson,et al.  A stabilized SQP method: superlinear convergence , 2017, Math. Program..

[16]  Brian W. Kernighan,et al.  AMPL: A Modeling Language for Mathematical Programming , 1993 .

[17]  R. Tyrrell Rockafellar,et al.  Augmented Lagrangians and Applications of the Proximal Point Algorithm in Convex Programming , 1976, Math. Oper. Res..

[18]  P. Gill,et al.  Sequential Quadratic Programming Methods , 2012 .

[19]  William W. Hager,et al.  Stabilized Sequential Quadratic Programming , 1999, Comput. Optim. Appl..

[20]  Michael P. Friedlander,et al.  A primal–dual regularized interior-point method for convex quadratic programs , 2010, Mathematical Programming Computation.

[21]  José Mario Martínez,et al.  Practical augmented Lagrangian methods for constrained optimization , 2014, Fundamentals of algorithms.

[22]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[23]  Klaus Schittkowski,et al.  Test examples for nonlinear programming codes , 1980 .

[24]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[25]  Damián R. Fernández,et al.  An inexact restoration strategy for the globalization of the sSQP method , 2012, Computational Optimization and Applications.

[26]  Alexey F. Izmailov,et al.  Globalizing Stabilized Sequential Quadratic Programming Method by Smooth Primal-Dual Exact Penalty Function , 2016, J. Optim. Theory Appl..

[27]  M. J. D. Powell,et al.  A fast algorithm for nonlinearly constrained optimization calculations , 1978 .

[28]  Dominique Orban,et al.  Iterative Solution of Symmetric Quasi-Definite Linear Systems , 2017 .

[29]  Stephen J. Wright.  An Algorithm for Degenerate Nonlinear Programming with Rapid Local Convergence , 2005, SIAM J. Optim..

[30]  Riadh Omheni,et al.  A globally and quadratically convergent primal–dual augmented Lagrangian algorithm for equality constrained optimization , 2017, Optim. Methods Softw..

[31]  Mikhail V. Solodov,et al.  Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems , 2010, Math. Program..

[32]  Paul T. Boggs,et al.  Sequential Quadratic Programming , 1995, Acta Numerica.

[33]  R. Vanderbei.  Symmetric Quasi-Definite Matrices , 2006 .

[34]  Matthias Heinkenschloss,et al.  A Matrix-Free Trust-Region SQP Method for Equality Constrained Optimization , 2014, SIAM J. Optim..

[35]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[36]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.

[37]  Daniel P. Robinson,et al.  A Globally Convergent Stabilized SQP Method , 2013, SIAM J. Optim..

[38]  Michael A. Saunders,et al.  LSMR: An Iterative Algorithm for Sparse Least-Squares Problems , 2010, SIAM J. Sci. Comput..

[39]  Dominique Orban,et al.  From global to local convergence of interior methods for nonlinear optimization , 2013, Optim. Methods Softw..

[40]  Michael A. Saunders,et al.  LSQR: An Algorithm for Sparse Linear Equations and Sparse Least Squares , 1982, TOMS.

[41]  Jorge Nocedal,et al.  Representations of quasi-Newton matrices and their use in limited memory methods , 1994, Math. Program..

[42]  Yin Zhang,et al.  An SQP Augmented Lagrangian BFGS Algorithm for Constrained Optimization , 1992, SIAM J. Optim..

[43]  Nicholas I. M. Gould,et al.  CUTEst: a Constrained and Unconstrained Testing Environment with safe threads for mathematical optimization , 2013, Computational Optimization and Applications.