Stabilized SQP revisited

The stabilized version of the sequential quadratic programming algorithm (sSQP) had been developed in order to achieve superlinear convergence in situations when the Lagrange multipliers associated to a solution are not unique. Within the framework of Fischer (Math Program 94:91–124, 2002), the key to local superlinear convergence of sSQP are the following two properties: upper Lipschitzian behavior of solutions of the Karush-Kuhn-Tucker (KKT) system under canonical perturbations and local solvability of sSQP subproblems with the associated primal-dual step being of the order of the distance from the current iterate to the solution set of the unperturbed KKT system. According to Fernández and Solodov (Math Program 125:47–73, 2010), both of these properties are ensured by the second-order sufficient optimality condition (SOSC) without any constraint qualification assumptions. In this paper, we state precise relationships between the upper Lipschitzian property of solutions of KKT systems, error bounds for KKT systems, the notion of critical Lagrange multipliers (a subclass of multipliers that violate SOSC in a very special way), the second-order necessary condition for optimality, and solvability of sSQP subproblems. Moreover, for the problem with equality constraints only, we prove superlinear convergence of sSQP under the assumption that the dual starting point is close to a noncritical multiplier. Since noncritical multipliers include all those satisfying SOSC but are not limited to them, we believe this gives the first superlinear convergence result for any Newtonian method for constrained optimization under assumptions that do not include any constraint qualifications and are weaker than SOSC. In the general case when inequality constraints are present, we show that such a relaxation of assumptions is not possible. We also consider applying sSQP to the problem where inequality constraints are reformulated into equalities using slack variables, and discuss the assumptions needed for convergence in this approach. We conclude with consequences for local regularization methods proposed in (Izmailov and Solodov SIAM J Optim 16:210–228, 2004; Wright SIAM J. Optim. 15:673–676, 2005). In particular, we show that these methods are still locally superlinearly convergent under the noncritical multiplier assumption, weaker than SOSC employed originally.

[1]  Andreas Fischer,et al.  Local behavior of an iterative framework for generalized equations with nonisolated solutions , 2002, Math. Program..

[2]  Alexey F. Izmailov,et al.  On attraction of Newton-type iterates to multipliers violating second-order sufficiency conditions , 2009, Math. Program..

[3]  Alexey F. Izmailov,et al.  Examples of dual behaviour of Newton-type methods on optimization problems with degenerate constraints , 2009, Comput. Optim. Appl..

[4]  Mikhail V. Solodov,et al.  Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems , 2010, Math. Program..

[5]  William W. Hager,et al.  Stability in the presence of degeneracy and error estimation , 1999, Math. Program..

[6]  Paul T. Boggs,et al.  Sequential Quadratic Programming , 1995, Acta Numerica.

[7]  A. F. Izmailov,et al.  Numerical results for a globalized active-set Newton method for mixed complementarity problems , 2005 .

[8]  William W. Hager,et al.  Stabilized Sequential Quadratic Programming , 1999, Comput. Optim. Appl..

[9]  Stephen J. Wright,et al.  Numerical Behavior of a Stabilized SQP Method for Degenerate NLP Problems , 2002, COCOS.

[10]  Stephen J. Wright.  Constraint identification and algorithm stabilization for degenerate nonlinear programs , 2000, Math. Program..

[11]  J. F. Bonnans,et al.  Local analysis of Newton-type methods for variational inequalities and nonlinear programming , 1994 .

[12]  Stephen J. Wright.  Superlinear Convergence of a Stabilized SQP Method to a Degenerate Solution , 1998, Comput. Optim. Appl..

[13]  Francisco Facchinei,et al.  On the Accurate Identification of Active Constraints , 1998, SIAM J. Optim..

[14]  Jerzy Kyparisis,et al.  On uniqueness of Kuhn-Tucker multipliers in nonlinear programming , 1985, Math. Program..

[15]  J. Frédéric Bonnans,et al.  Perturbation Analysis of Optimization Problems , 2000, Springer Series in Operations Research.

[16]  Alexey F. Izmailov,et al.  Newton-Type Methods for Optimization Problems without Constraint Qualifications , 2004, SIAM J. Optim..

[17]  Alexey F. Izmailov,et al.  On attraction of linearly constrained Lagrangian methods and of stabilized and quasi-Newton SQP methods to critical multipliers , 2011, Math. Program..

[18]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[19]  A. F. Izmailov.  Solution sensitivity for Karush–Kuhn–Tucker systems with non-unique Lagrange multipliers , 2010 .

[20]  Paul Pinsler.  Über das Vorkommen definiter und semidefiniter Formen in Scharen quadratischer Formen , 1936 .

[21]  G. Debreu.  Definite and Semidefinite Quadratic Forms , 1952 .

[22]  Stephen J. Wright.  An Algorithm for Degenerate Nonlinear Programming with Rapid Local Convergence , 2005, SIAM J. Optim..

[23]  Stephen J. Wright.  Modifying SQP for Degenerate Problems , 2002, SIAM J. Optim..

[24]  O. Mangasarian,et al.  The Fritz John Necessary Optimality Conditions in the Presence of Equality and Inequality Constraints , 1967 .

[25]  Alexey F. Izmailov,et al.  On the analytical and numerical stability of critical Lagrange multipliers , 2005 .

[26]  Alexey F. Izmailov,et al.  Sharp Primal Superlinear Convergence Results for Some Newtonian Methods for Constrained Optimization , 2010, SIAM J. Optim..