An inertial forward–backward algorithm for the minimization of the sum of two nonconvex functions

We propose a forward–backward proximal-type algorithm with inertial/memory effects for minimizing the sum of a nonsmooth function with a smooth one in the nonconvex setting. Every sequence of iterates generated by the algorithm converges to a critical point of the objective function provided an appropriate regularization of the objective satisfies the Kurdyka-Łojasiewicz inequality, which is for instance fulfilled for semi-algebraic functions. We illustrate the theoretical results by considering two numerical experiments: the first one concerns the ability of recovering the local optimal solutions of nonconvex optimization problems, while the second one refers to the restoration of a noisy blurred image.

[1]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[2]  K. Kurdyka.  On gradients of functions definable in o-minimal structures , 1998 .

[3]  Felipe Alvarez,et al.  On the Minimizing Property of a Second Order Dissipative System in Hilbert Spaces , 2000, SIAM J. Control. Optim..

[4]  H. Attouch,et al.  An Inertial Proximal Method for Maximal Monotone Operators via Discretization of a Nonlinear Oscillator with Damping , 2001 .

[5]  A. Moudafi,et al.  Convergence of a splitting inertial proximal method for monotone operators , 2003 .

[6]  P. L. Combettes,et al.  Solving monotone inclusions via compositions of nonexpansive averaged operators , 2004 .

[7]  Felipe Alvarez,et al.  Weak Convergence of a Relaxed and Inertial Hybrid Projection-Proximal Point Algorithm for Maximal Monotone Operators in Hilbert Space , 2003, SIAM J. Optim..

[8]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[9]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[10]  Adrian S. Lewis,et al.  Clarke Subgradients of Stratifiable Functions , 2006, SIAM J. Optim..

[11]  Paul-Emile Maingé,et al.  Convergence of New Inertial Proximal Methods for DC Programming , 2008, SIAM J. Optim..

[12]  P. Maingé.  Convergence theorems for inertial KM-type algorithms , 2008 .

[13]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[14]  J. Bolte,et al.  Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity , 2009 .

[15]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[16]  Boris Polyak,et al.  B.S. Mordukhovich. Variational Analysis and Generalized Differentiation. I. Basic Theory, II. Applications , 2009 .

[17]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[18]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[19]  A. Cabot,et al.  Asymptotics for Some Proximal-like Method Involving Inertia and Memory Aspects , 2011 .

[20]  J. Pesquet,et al.  A Parallel Inertial Proximal Optimization Method , 2012 .

[21]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[22]  Fengming Ma,et al.  An Implementable First-Order Primal-Dual Algorithm for Structured Convex Optimization , 2014 .

[23]  Matthew K. Tam,et al.  Proximal Heterogeneous Block Input-Output Method and application to Blind Ptychographic Diffraction Imaging , 2014, 1408.1887.

[24]  R. Boţ,et al.  A Hybrid Proximal-Extragradient Algorithm with Inertial Effects , 2014, 1407.0214.

[25]  Shiqian Ma,et al.  A general inertial proximal point method for mixed variational inequality problem , 2014, 1407.8238.

[26]  Émilie Chouzenoux,et al.  Variable Metric Forward–Backward Algorithm for Minimizing the Sum of a Differentiable Function and a Convex Function , 2013, Journal of Optimization Theory and Applications.

[27]  Juan Peypouquet,et al.  A Dynamical Approach to an Inertial Forward-Backward Algorithm for Convex Minimization , 2014, SIAM J. Optim..

[28]  R. Boţ,et al.  An inertial alternating direction method of multipliers , 2014, 1404.4582.

[29]  Shiqian Ma,et al.  Inertial primal-dual algorithms for structured convex optimization , 2014, 1409.2992.

[30]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[31]  Thomas Brox,et al.  iPiano: Inertial Proximal Algorithm for Nonconvex Optimization , 2014, SIAM J. Imaging Sci..

[32]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[33]  Juan Peypouquet,et al.  Splitting Methods with Variable Metric for Kurdyka–Łojasiewicz Functions and General Convergence Rates , 2015, J. Optim. Theory Appl..

[34]  Caihua Chen,et al.  A General Inertial Proximal Point Algorithm for Mixed Variational Inequality Problem , 2015, SIAM J. Optim..

[35]  Radu Ioan Bot,et al.  Inertial Douglas-Rachford splitting for monotone inclusion problems , 2014, Appl. Math. Comput..

[36]  Shoham Sabach,et al.  Proximal Heterogeneous Block Implicit-Explicit Method and Application to Blind Ptychographic Diffraction Imaging , 2015, SIAM J. Imaging Sci..

[37]  Radu Ioan Bot,et al.  An inertial forward-backward-forward primal-dual splitting algorithm for solving monotone inclusion problems , 2014, Numerical Algorithms.

[38]  Radu Ioan Bot,et al.  An Inertial Tseng’s Type Proximal Algorithm for Nonsmooth and Nonconvex Optimization Problems , 2014, J. Optim. Theory Appl..