The value function approach to convergence analysis in composite optimization

This works aims at understanding further convergence properties of first order local search methods with complex geometries. We focus on the composite optimization model which unifies within a simple formalism many problems of this type. We provide a general convergence analysis of the composite Gauss-Newton method under tameness assumptions (an extension of semi-algebraicity). Tameness is a very general condition satisfied by virtually all problems solved in practice. The analysis is based on recent progresses in understanding convergence properties of sequential convex programming methods through the value function.

[1]  Chong Li,et al.  On convergence of the Gauss-Newton method for convex composite optimization , 2002, Math. Program..

[2]  L. Dries,et al.  Geometric categories and o-minimal structures , 1996 .

[3]  D. Noll,et al.  Convergence of Linesearch and Trust-Region Methods Using the Kurdyka–Łojasiewicz Inequality , 2013 .

[4]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[5]  M. Coste.  AN INTRODUCTION TO O-MINIMAL GEOMETRY , 2002 .

[6]  Michael A. Saunders,et al.  SNOPT: An SQP Algorithm for Large-Scale Constrained Optimization , 2002, SIAM J. Optim..

[7]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[8]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[9]  Robert E. Mahony,et al.  Convergence of the Iterates of Descent Methods for Analytic Cost Functions , 2005, SIAM J. Optim..

[10]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[11]  K. Kurdyka.  On gradients of functions definable in o-minimal structures , 1998 .

[12]  Edouard Pauwels,et al.  Majorization-Minimization Procedures and Convergence of SQP Methods for Semi-Algebraic and Tame Programs , 2014, Math. Oper. Res..

[13]  Chong Li,et al.  Majorizing Functions and Convergence of the Gauss--Newton Method for Convex Composite Optimization , 2007, SIAM J. Optim..

[14]  NG K.F..  MAJORIZING FUNCTIONS AND CONVERGENCE OF THE GAUSS–NEWTON METHOD FOR CONVEX COMPOSITE OPTIMIZATION∗ , 2007 .

[15]  丸山 徹.  Convex Analysisの二,三の進展について , 1977 .

[16]  Adrian S. Lewis,et al.  Clarke Subgradients of Stratifiable Functions , 2006, SIAM J. Optim..

[17]  James V. Burke,et al.  Descent methods for composite nondifferentiable optimization problems , 1985, Math. Program..

[18]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[19]  Alfred Auslender,et al.  An Extended Sequential Quadratically Constrained Quadratic Programming Algorithm for Nonlinear, Semidefinite, and Second-Order Cone Programming , 2013, J. Optim. Theory Appl..

[20]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[21]  Lou van den Dries.  o-Minimal structures , 1996 .

[22]  Stephen J. Wright,et al.  A proximal method for composite minimization , 2008, Mathematical Programming.

[23]  Michael C. Ferris,et al.  A Gauss—Newton method for convex composite optimization , 1995, Math. Program..

[24]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..