Convergence analysis of the Gauss-Newton method for convex inclusion and convex-composite optimization problems

Abstract Using the convex process theory we study the convergence issues of the iterative sequences generated by the Gauss–Newton method for the convex inclusion problem defined by a cone C and a smooth function F (the derivative is denoted by F ′ ). The restriction in our consideration is minimal and, even in the classical case (the initial point x 0 is assumed to satisfy the following two conditions: F ′ is Lipschitz around x 0 and the convex process T x 0 , defined by T x 0 ⋅ = F ′ ( x 0 ) ⋅ − C , is surjective), our results are new in giving sufficient conditions (which are weaker than the known ones and have a remarkable property being affine-invariant) ensuring the convergence of the iterative sequence with initial point x 0 . The same study is also made for the so-called convex-composite optimization problem (with objective function given as the composite of a convex function with a smooth map).

[1]  J. V. Burke,et al.  Optimality conditions for non-finite valued convex composite functions , 1992, Math. Program..

[2]  R. Rockafellar First- and second-order epi-differentiability in nonlinear programming , 1988 .

[3]  Vaithilingam Jeyakumar,et al.  Complete characterizations of stable Farkas’ lemma and cone-convex programming duality , 2008, Math. Program..

[4]  James V. Burke,et al.  Descent methods for composite nondifferentiable optimization problems , 1985, Math. Program..

[5]  R. Rockafellar,et al.  Implicit Functions and Solution Mappings , 2009 .

[6]  M. R. Osborne,et al.  Strong uniqueness and second order convergence in nonlinear discrete approximation , 1980 .

[7]  P. Deuflhard Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms , 2011 .

[8]  Roger Fletcher,et al.  Practical methods of optimization; (2nd ed.) , 1987 .

[9]  Chong Li,et al.  Majorizing Functions and Convergence of the Gauss--Newton Method for Convex Composite Optimization , 2007, SIAM J. Optim..

[10]  Michael C. Ferris,et al.  A Gauss—Newton method for convex composite optimization , 1995, Math. Program..

[11]  Boris S. Mordukhovich,et al.  Subdifferentials of value functions and optimality conditions for DC and bilevel infinite and semi-infinite programs , 2010, Math. Program..

[12]  R. Rockafellar,et al.  Implicit Functions and Solution Mappings: A View from Variational Analysis , 2009 .

[13]  R. Fletcher,et al.  Second order corrections for non-differentiable optimization , 1982 .

[14]  P. Deuflhard,et al.  Affine Invariant Convergence Theorems for Newton’s Method and Extensions to Related Methods , 1979 .

[15]  Robert S. Womersley,et al.  Local properties of algorithms for minimizing nonsmooth composite functions , 1985, Math. Program..

[16]  Marco A. López,et al.  On the Stability of the Feasible Set in Optimization Problems , 2010, SIAM J. Optim..

[17]  Chong Li,et al.  On convergence of the Gauss-Newton method for convex composite optimization , 2002, Math. Program..

[18]  R. Tapia,et al.  Optimal Error Bounds for the Newton–Kantorovich Theorem , 1974 .

[19]  R. Tyrrell Rockafellar,et al.  Convex Analysis , 1970, Princeton Landmarks in Mathematics and Physics.

[20]  Vaithilingam Jeyakumar,et al.  Inequality systems and global optimization , 1996 .

[21]  R. Tyrrell Rockafellar,et al.  Monotone processes of convex and concave type , 1967 .

[22]  A. Ostrowski Solution of equations in Euclidean and Banach spaces , 1973 .

[23]  T. T. A. Nghia,et al.  Farkas-type results and duality for DC programs with convex constraints , 2007 .

[24]  Boris S. Mordukhovich,et al.  Variational Analysis in Semi-Infinite and Infinite Programming, II: Necessary Optimality Conditions , 2010, SIAM J. Optim..

[25]  S. M. Robinson Extension of Newton's method to nonlinear functions with values in a cone , 1972 .

[26]  Jean-Pierre Dedieu,et al.  Approximate Solutions of Analytic Inequality Systems , 2000, SIAM J. Optim..

[27]  S. M. Robinson Normed convex processes , 1972 .

[28]  Vladimir M. Veliov,et al.  Metric Regularity of Newton's Iteration , 2011, SIAM J. Control. Optim..

[29]  Boris S. Mordukhovich,et al.  Variational Analysis in Semi-Infinite and Infinite Programming, I: Stability of Linear Inequality Systems of Feasible Solutions , 2009, SIAM J. Optim..