An LP-Newton method: nonsmooth equations, KKT systems, and nonisolated solutions

We define a new Newton-type method for the solution of constrained systems of equations and analyze in detail its properties. Under suitable conditions, that do not include differentiability or local uniqueness of solutions, the method converges locally quadratically to a solution of the system, thus filling an important gap in the existing theory. The new algorithm improves on known methods and, when particularized to KKT systems derived from optimality conditions for constrained optimization or variational inequalities, it has theoretical advantages even over methods specifically designed to solve such systems.

[1]  Alexey F. Izmailov,et al.  Newton-Type Methods for Optimization Problems without Constraint Qualifications , 2004, SIAM J. Optim..

[2]  Liqun Qi,et al.  On the convergence of an inexact Newton-type method , 2006, Oper. Res. Lett..

[3]  Francisco Facchinei,et al.  On the Accurate Identification of Active Constraints , 1998, SIAM J. Optim..

[4]  M. Fukushima,et al.  Levenberg–Marquardt methods with strong local convergence properties for solving nonlinear equations with convex constraints , 2004 .

[5]  Francisco Facchinei,et al.  Generalized Nash Equilibrium Problems , 2010, Ann. Oper. Res..

[6]  Stephen J. Wright.  An Algorithm for Degenerate Nonlinear Programming with Rapid Local Convergence , 2005, SIAM J. Optim..

[7]  Stephen J. Wright,et al.  Active Set Identification in Nonlinear Programming , 2006, SIAM J. Optim..

[8]  Ya-Xiang Yuan,et al.  On the Quadratic Convergence of the Levenberg-Marquardt Method without Nonsingularity Assumption , 2005, Computing.

[9]  Andreas Fischer,et al.  Modified Wilson's Method for Nonlinear Programs with Nonunique Multipliers , 1999, Math. Oper. Res..

[10]  Stephen J. Wright.  Superlinear Convergence of a Stabilized SQP Method to a Degenerate Solution , 1998, Comput. Optim. Appl..

[11]  Alexey F. Izmailov,et al.  Stabilized SQP revisited , 2012, Math. Program..

[12]  Defeng Sun,et al.  Solving Karush--Kuhn--Tucker Systems via the Trust Region and the Conjugate Gradient Methods , 2003, SIAM J. Optim..

[13]  D. Klatte.  Nonsmooth equations in optimization , 2002 .

[14]  P. Tseng.  Growth behavior of a class of merit functions for the nonlinear complementarity problem , 1996 .

[15]  M. Fukushima,et al.  On the Rate of Convergence of the Levenberg-Marquardt Method , 2001 .

[16]  Pradyumn Kumar Shukla,et al.  Levenberg-Marquardt Algorithms for Nonlinear Equations, Multi-objective Optimization, and Complementarity Problems , 2009 .

[17]  J. Frédéric Bonnans,et al.  Perturbation Analysis of Optimization Problems , 2000, Springer Series in Operations Research.

[18]  Alexey F. Izmailov,et al.  The Josephy–Newton Method for Semismooth Generalized Equations and Semismooth SQP for Optimization , 2013 .

[19]  Francisco Facchinei,et al.  On the Identification of Zero Variables in an Interior-Point Framework , 1999, SIAM J. Optim..

[20]  Masao Fukushima,et al.  A Superlinearly Convergent Algorithm for the Monotone Nonlinear Complementarity Problem Without Uniqueness and Nondegeneracy Conditions , 2002, Math. Oper. Res..

[21]  Michael Ulbrich,et al.  Nonmonotone Trust-Region Methods for Bound-Constrained Semismooth Equations with Applications to Nonlinear Mixed Complementarity Problems , 2000, SIAM J. Optim..

[22]  Liqun Qi,et al.  Convergence Analysis of Some Algorithms for Solving Nonsmooth Equations , 1993, Math. Oper. Res..

[23]  B. Kummer.  NEWTON's METHOD FOR NON-DIFFERENTIABLE FUNCTIONS , 1988, Advances in Mathematical Optimization.

[24]  F. Luque.  Asymptotic convergence analysis of the proximal point algorithm , 1984 .

[25]  F. Facchinei,et al.  Finite-Dimensional Variational Inequalities and Complementarity Problems , 2003 .

[26]  Andreas Fischer,et al.  A Levenberg-Marquardt algorithm for unconstrained multicriteria optimization , 2008, Oper. Res. Lett..

[27]  Andreas Fischer,et al.  A unified local convergence analysis of inexact constrained Levenberg–Marquardt methods , 2011, Optimization Letters.

[28]  Andreas Fischer,et al.  A Framework for Analyzing Local Convergence Properties with Applications to Proximal-Point Algorithms , 2006 .

[29]  A. Fischer,et al.  On the inexactness level of robust Levenberg–Marquardt methods , 2010 .

[30]  Francisco Facchinei,et al.  A family of Newton methods for nonsmooth constrained systems with nonisolated solutions , 2013, Math. Methods Oper. Res..

[31]  A. Hoffman.  On approximate solutions of systems of linear inequalities , 1952 .

[32]  Andreas Fischer,et al.  Solution of monotone complementarity problems with locally Lipschitzian functions , 1997, Math. Program..

[33]  C. Kanzow.  Levenberg-Marquardt methods for constrained nonlinear equations with strong local convergence properties , 2004 .

[34]  Mikhail V. Solodov,et al.  Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems , 2010, Math. Program..

[35]  Diethard Klatte,et al.  Nonsmooth Equations in Optimization: "Regularity, Calculus, Methods And Applications" , 2006 .

[36]  Francisco Facchinei,et al.  Generalized Nash equilibrium problems and Newton methods , 2008, Math. Program..

[37]  William W. Hager,et al.  Stabilized Sequential Quadratic Programming , 1999, Comput. Optim. Appl..

[38]  F. Facchinei,et al.  A Simply Constrained Optimization Reformulation of KKT Systems Arising from Variational Inequalities , 1999 .

[39]  S. Scholtes.  Introduction to Piecewise Differentiable Equations , 2012 .

[40]  Christian Kanzow,et al.  Strictly feasible equation-based methods for mixed complementarity problems , 2001, Numerische Mathematik.

[41]  Liqun Qi,et al.  A nonsmooth version of Newton's method , 1993, Math. Program..

[42]  K. Ng,et al.  Error Bounds of Constrained Quadratic Functions and Piecewise Affine Inequality Systems , 2003 .

[43]  Andreas Fischer,et al.  Local behavior of an iterative framework for generalized equations with nonisolated solutions , 2002, Math. Program..

[44]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[45]  Stephen J. Wright.  Constraint identification and algorithm stabilization for degenerate nonlinear programs , 2000, Math. Program..

[46]  Francisco Facchinei,et al.  A nonsmooth inexact Newton method for the solution of large-scale nonlinear complementarity problems , 1997, Math. Program..