Semismooth Karush-Kuhn-Tucker Equations and Convergence Analysis of Newton and Quasi-Newton Methods for Solving these Equations

There are several forms of systems of nonsmooth equations which are equivalent to the Karush-Kuhn-Tucker KKT system of a nonlinearly constrained optimization problem NLP. If the NLP is twice continuously differentiable and the Hessian functions of its objective and constraint functions are locally Lipschitzian, then these KKT equations are strongly semismooth. If furthermore the linear independence condition and the strong second-order sufficiency condition are satisfied at a KKT point, then the generalized Jacobians of these KKT equations are nonsingular at that point and the sequence generated by the generalized Newton method converges to this point Q-quadratically. However, direct application of quasi-Newton methods cannot guarantee Q-superlinear convergence. We present a mixed quasi-Newton method which converges Q-superlinearly with common symmetrical updating rules under the above conditions for the generalized Newton method. Superlinear convergence of the primal variables and global convergence are also discussed.

[1]  Jong-Shi Pang,et al.  A B-differentiable equation-based, globally and locally quadratically convergent algorithm for nonlinear programs, complementarity and variational inequality problems , 1991, Math. Program..

[2]  Patrick T. Harker,et al.  Finite-dimensional variational inequality and nonlinear complementarity problems: A survey of theory, algorithms and applications , 1990, Math. Program..

[3]  Anthony V. Fiacco,et al.  Nonlinear programming;: Sequential unconstrained minimization techniques , 1968 .

[4]  Stephen M. Robinson,et al.  Nonsingularity and symmetry for linear normal maps , 1993, Math. Program..

[5]  Xiaojun Chen,et al.  A parameterized Newton method and a quasi-Newton method for nonsmooth equations , 1994, Comput. Optim. Appl..

[6]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[7]  Masao Fukushima,et al.  Equivalent differentiable optimization problems and descent methods for asymmetric variational inequality problems , 1992, Math. Program..

[8]  Jong-Shi Pang,et al.  A degree-theoretic approach to parametric nonsmooth equations with multivalued perturbed solution sets , 1993, Math. Program..

[9]  R. Mifflin.  Semismooth and Semiconvex Functions in Constrained Optimization , 1977 .

[10]  Andreas Fischer,et al.  Solution of monotone complementarity problems with locally Lipschitzian functions , 1997, Math. Program..

[11]  Bernd Kummer.  On stability and newton-type methods for lipschitzian equations with applications to optimization problems , 1992 .

[12]  J. M. Martínez,et al.  Inexact Newton methods for solving nonsmooth equations , 1995 .

[13]  John E. Dennis,et al.  Numerical methods for unconstrained optimization and nonlinear equations , 1983, Prentice Hall series in computational mathematics.

[14]  A. Fischer.  An NCP–Function and its Use for the Solution of Complementarity Problems , 1995 .

[15]  R. Tyrrell Rockafellar,et al.  Proximal Subgradients, Marginal Values, and Augmented Lagrangians in Nonconvex Optimization , 1981, Math. Oper. Res..

[16]  L. Qi,et al.  A Globally Convergent Successive Approximation Method for Severely Nonsmooth Equations , 1995 .

[17]  Jong-Shi Pang,et al.  NE/SQP: A robust algorithm for the nonlinear complementarity problem , 1993, Math. Program..

[18]  Eugene L. Allgower,et al.  Numerical continuation methods - an introduction , 1990, Springer series in computational mathematics.

[19]  Masao Fukushima,et al.  Equivalence of Complementarity Problems to Differentiable Minimization: A Unified Approach , 1996, SIAM J. Optim..

[20]  Masao Fukushima,et al.  A globally convergent Newton method for solving strongly monotone variational inequalities , 1993, Math. Program..

[21]  Jong-Shi Pang,et al.  Nonsmooth Equations: Motivation and Algorithms , 1993, SIAM J. Optim..

[22]  J. Pang,et al.  A Trust Region Method for Constrained Nonsmooth Equations , 1994 .

[23]  Andrzej P. Wierzbicki.  Note on the equivalence of Kuhn-Tucker complementarity conditions to an equation , 1982 .

[24]  S. Smale.  Convergent process of price adjust-ment and global newton methods , 1976 .

[25]  Jong-Shi Pang,et al.  Globally Convergent Newton Methods for Nonsmooth Equations , 1992, Math. Oper. Res..

[26]  Jong-Shi Pang,et al.  An inexact NE/SQP method for solving the nonlinear complementarity problem , 1992, Comput. Optim. Appl..

[27]  Jong-Shi Pang,et al.  Newton's Method for B-Differentiable Equations , 1990, Math. Oper. Res..

[28]  Olvi L. Mangasarian,et al.  Superlinearly convergent quasi-newton algorithms for nonlinearly constrained optimization problems , 1976, Math. Program..

[29]  S. M. Robinson.  Generalized equations and their solutions, part II: Applications to nonlinear programming , 1982 .

[30]  B. Kummer.  NEWTON's METHOD FOR NON-DIFFERENTIABLE FUNCTIONS , 1988, Advances in Mathematical Optimization.

[31]  M. Kojima.  Strongly Stable Stationary Solutions in Nonlinear Programs. , 1980 .

[32]  J. F. Bonnans,et al.  Local analysis of Newton-type methods for variational inequalities and nonlinear programming , 1994 .

[33]  Liqun Qi,et al.  A nonsmooth version of Newton's method , 1993, Math. Program..

[34]  M. J. D. Powell,et al.  Variable Metric Methods for Constrained Optimization , 1982, ISMP.

[35]  K. Christian,et al.  A Class of Netwton-Type methods for equality and ineqality constrained optimization , 1995 .

[36]  A. Fischer.  A special newton-type optimization method , 1992 .

[37]  O. Mangasarian.  Equivalence of the Complementarity Problem to a System of Nonlinear Equations , 1976 .

[38]  J. Abadie,et al.  Generalized reduced gradient and global Newton methods , 1986 .

[39]  Liqun Qi,et al.  Trust Region Algorithms for Solving Nonsmooth Equations , 1995, SIAM J. Optim..

[40]  B. Kummer.  Newton’s Method Based on Generalized Derivatives for Nonsmooth Functions: Convergence Analysis , 1992 .

[41]  Liqun Qi,et al.  Convergence Analysis of Some Algorithms for Solving Nonsmooth Equations , 1993, Math. Oper. Res..

[42]  Shih-Ping Han,et al.  Superlinearly convergent variable metric algorithms for general nonlinear programming problems , 1976, Math. Program..

[43]  José Mario Martínez,et al.  Least change update methods for nonlinear systems with nondifferentiable terms , 1993 .

[44]  Klaus Schönefeld,et al.  Newton-type methods for nonlinearly constrained programming problems-algorithms and theory , 1988 .

[45]  Jong-Shi Pang,et al.  Serial and Parallel Computation of Karush-Kuhn-Tucker Points via Nonsmooth Equations , 1994, SIAM J. Optim..

[46]  Jerzy Kyparisis,et al.  Local convergence of quasi-Newton methods for B-differentiable equations , 1992, Math. Program..

[47]  Jershan Chiang,et al.  Convergence Rates for the Variable, the Multiplier, and the Pair in SQP Methods , 1989 .

[48]  O. Mangasarian.  Unconstrained Lagrangians in Nonlinear Programming , 1975 .

[49]  J. J. Moré,et al.  A Characterization of Superlinear Convergence and its Application to Quasi-Newton Methods , 1973 .

[50]  J. J. Moré,et al.  Quasi-Newton Methods, Motivation and Theory , 1974 .

[51]  Daniel Ralph,et al.  Global Convergence of Damped Newton's Method for Nonsmooth Equations via the Path Search , 1994, Math. Oper. Res..

[52]  Olvi L. Mangasarian,et al.  Nonlinear complementarity as unconstrained and constrained minimization , 1993, Math. Program..

[53]  Liqun Qi,et al.  Superlinearly convergent approximate Newton methods for LC1 optimization problems , 1994, Math. Program..

[54]  J. F. Bonnans.  Local study of Newton type algorithms for constrained problems , 1988 .