A forward-backward dynamical approach to the minimization of the sum of a nonsmooth convex with a smooth nonconvex function

We address the minimization of the sum of a proper, convex and lower semicontinuous with a (possibly nonconvex) smooth function from the perspective of an implicit dynamical system of forward-backward type. The latter is formulated by means of the gradient of the smooth function and of the proximal point operator of the nonsmooth one. The trajectory generated by the dynamical system is proved to asymptotically converge to a critical point of the objective, provided a regularization of the latter satisfies the Kurdyka-Łojasiewicz property. Convergence rates for the trajectory in terms of the Łojasiewicz exponent of the regularized objective function are also provided.

[1]  Radu Ioan Bot,et al.  Second Order Forward-Backward Dynamical Systems For Monotone Inclusion Problems , 2015, SIAM J. Control. Optim..

[2]  Radu Ioan Bot,et al.  An inertial forward–backward algorithm for the minimization of the sum of two nonconvex functions , 2014, EURO J. Comput. Optim..

[3]  Radu Ioan Bot,et al.  An Inertial Tseng’s Type Proximal Algorithm for Nonsmooth and Nonconvex Optimization Problems , 2014, J. Optim. Theory Appl..

[4]  Juan Peypouquet,et al.  Splitting Methods with Variable Metric for Kurdyka–Łojasiewicz Functions and General Convergence Rates , 2015, J. Optim. Theory Appl..

[5]  R. Boţ,et al.  Convergence rates for forward-backward dynamical systems associated with strongly monotone inclusions , 2015, 1504.01863.

[6]  Radu Ioan Bot,et al.  A forward-backward-forward differential equation and its asymptotic properties , 2015, 1503.07728.

[7]  R. Boţ,et al.  Approaching the solving of constrained variational inequalities via penalty term-based dynamical systems , 2015, 1503.01871.

[8]  Shoham Sabach,et al.  Proximal Heterogeneous Block Implicit-Explicit Method and Application to Blind Ptychographic Diffraction Imaging , 2015, SIAM J. Imaging Sci..

[9]  B. Svaiter,et al.  A dynamic approach to a proximal-Newton method for monotone inclusions in Hilbert spaces, with complexity O(1/n^2) , 2015, 1502.04286.

[10]  Ernö Robert Csetnek,et al.  A Dynamical System Associated with the Fixed Points Set of a Nonexpansive Operator , 2014, 1411.4442.

[11]  Giorgio C. Buttazzo,et al.  Variational Analysis in Sobolev and BV Spaces - Applications to PDEs and Optimization, Second Edition , 2014, MPS-SIAM series on optimization.

[12]  Matthew K. Tam,et al.  Proximal Heterogeneous Block Input-Output Method and application to Blind Ptychographic Diffraction Imaging , 2014, 1408.1887.

[13]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[14]  Émilie Chouzenoux,et al.  Variable Metric Forward–Backward Algorithm for Minimizing the Sum of a Differentiable Function and a Convex Function , 2013, Journal of Optimization Theory and Applications.

[15]  Benar Fux Svaiter,et al.  Newton-Like Dynamics and Forward-Backward Methods for Structured Monotone Inclusions in Hilbert Spaces , 2014, J. Optim. Theory Appl..

[16]  Thomas Brox,et al.  iPiano: Inertial Proximal Algorithm for Nonconvex Optimization , 2014, SIAM J. Imaging Sci..

[17]  H. Attouch,et al.  Dynamical systems and forward–backward algorithms associated with the sum of a convex subdifferential and a monotone cocoercive operator , 2014, 1403.6312.

[18]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[19]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[20]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[21]  Benar Fux Svaiter,et al.  A Continuous Dynamical Newton-Like Approach to Solving Monotone Inclusions , 2011, SIAM J. Control. Optim..

[22]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[23]  J. Bolte,et al.  Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity , 2009 .

[24]  Boris Polyak,et al.  B.S. Mordukhovich. Variational Analysis and Generalized Differentiation. I. Basic Theory, II. Applications , 2009 .

[25]  H. Attouch,et al.  Asymptotic behavior of coupled dynamical systems with multiscale aspects , 2009, 0904.0397.

[26]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[27]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[28]  Adrian S. Lewis,et al.  Clarke Subgradients of Stratifiable Functions , 2006, SIAM J. Optim..

[29]  B. Mordukhovich.  Variational Analysis and Generalized Differentiation II: Applications , 2006 .

[30]  Felipe Alvarez,et al.  Weak Convergence of a Relaxed and Inertial Hybrid Projection-Proximal Point Algorithm for Maximal Monotone Operators in Hilbert Space , 2003, SIAM J. Optim..

[31]  J. Bolte.  Continuous Gradient Projection Method in Hilbert Spaces , 2003 .

[32]  A. Antipin,et al.  MINIMIZATION OF CONVEX FUNCTIONS ON CONVEX SETS BY MEANS OF DIFFERENTIAL EQUATIONS , 2003 .

[33]  Heinz H. Bauschke,et al.  Phase retrieval, error reduction algorithm, and Fienup variants: a view from convex optimization. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[34]  J. Bolte,et al.  A second-order gradient-like dissipative dynamical system with Hessian-driven damping.: Application to optimization and mechanics , 2002 .

[35]  øöö Blockinø.  Phase retrieval, error reduction algorithm, and Fienup variants: A view from convex optimization , 2002 .

[36]  H. Attouch,et al.  An Inertial Proximal Method for Maximal Monotone Operators via Discretization of a Nonlinear Oscillator with Damping , 2001 .

[37]  Felipe Alvarez,et al.  On the Minimizing Property of a Second Order Dissipative System in Hilbert Spaces , 2000, SIAM J. Control. Optim..

[38]  H. Attouch,et al.  THE HEAVY BALL WITH FRICTION METHOD, I. THE CONTINUOUS DYNAMICAL SYSTEM: GLOBAL EXPLORATION OF THE LOCAL MINIMA OF A REAL-VALUED FUNCTION BY ASYMPTOTIC ANALYSIS OF A DISSIPATIVE DYNAMICAL SYSTEM , 2000 .

[39]  H. Attouch,et al.  The heavy ball with friction dynamical system for convex constrained minimization problems , 2000 .

[40]  A. Haraux,et al.  Convergence of Solutions of Second-Order Gradient-Like Systems with Analytic Nonlinearities , 1998 .

[41]  K. Kurdyka.  On gradients of functions definable in o-minimal structures , 1998 .

[42]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[43]  A. Haraux,et al.  Systèmes dynamiques dissipatifs et applications , 1991 .

[44]  L. Simon.  Asymptotics for a class of non-linear evolution equations, with applications to geometric problems , 1983 .

[45]  H. Brezis.  Opérateurs maximaux monotones et semi-groupes de contractions dans les espaces de Hilbert , 1973 .