A Proximal Minimization Algorithm for Structured Nonconvex and Nonsmooth Problems

We propose a proximal algorithm for minimizing objective functions consisting of three summands: the composition of a nonsmooth function with a linear operator, another nonsmooth function, each of the nonsmooth summands depending on an independent block variable, and a smooth function which couples the two block variables. The algorithm is a full splitting method, which means that the nonsmooth functions are processed via their proximal operators, the smooth function via gradient steps, and the linear operator via matrix times vector multiplication. We provide sufficient conditions for the boundedness of the generated sequence and prove that any cluster point of the latter is a KKT point of the minimization problem. In the setting of the Kurdyka-\L{}ojasiewicz property we show global convergence, and derive convergence rates for the iterates in terms of the \L{}ojasiewicz exponent.

[1]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[2]  Shiqian Ma,et al.  Structured nonconvex and nonsmooth optimization: algorithms and iteration complexity analysis , 2016, Computational Optimization and Applications.

[3]  Radu Ioan Bot,et al.  An inertial forward–backward algorithm for the minimization of the sum of two nonconvex functions , 2014, EURO J. Comput. Optim..

[4]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[5]  Guoyin Li,et al.  Global Convergence of Splitting Methods for Nonconvex Composite Optimization , 2014, SIAM J. Optim..

[6]  B. Mordukhovich.  Variational Analysis and Generalized Differentiation II: Applications , 2006 .

[7]  Warren Hare,et al.  Computing proximal points of nonconvex functions , 2008, Math. Program..

[8]  J. Moreau.  Fonctions convexes duales et points proximaux dans un espace hilbertien , 1962 .

[9]  Thomas Pock,et al.  Inertial Proximal Alternating Linearized Minimization (iPALM) for Nonconvex and Nonsmooth Problems , 2016, SIAM J. Imaging Sci..

[10]  Radu Ioan Bot,et al.  The Proximal Alternating Direction Method of Multipliers in the Nonconvex Setting: Convergence Analysis and Rates , 2018, Math. Oper. Res..

[11]  Hédy Attouch,et al.  Proximal Alternating Minimization and Projection Methods for Nonconvex Problems: An Approach Based on the Kurdyka-Lojasiewicz Inequality , 2008, Math. Oper. Res..

[12]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[13]  K. Kurdyka.  On gradients of functions definable in o-minimal structures , 1998 .

[14]  F. Giannessi.  Variational Analysis and Generalized Differentiation , 2006 .

[15]  Adrian S. Lewis,et al.  Clarke Subgradients of Stratifiable Functions , 2006, SIAM J. Optim..

[16]  Qinghua Liu,et al.  Linearized ADMM for Nonconvex Nonsmooth Optimization With Convergence Analysis , 2017, IEEE Access.

[17]  Zongben Xu,et al.  Convergence of multi-block Bregman ADMM for nonconvex composite problems , 2015, Science China Information Sciences.

[18]  J. Bolte,et al.  Characterizations of Lojasiewicz inequalities: Subgradient flows, talweg, convexity , 2009 .

[19]  Zhi-Quan Luo,et al.  Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[21]  Adrian S. Lewis,et al.  Alternating Projections on Manifolds , 2008, Math. Oper. Res..

[22]  Zhi-Quan Luo,et al.  On the linear convergence of the alternating direction method of multipliers , 2012, Mathematical Programming.

[23]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.

[24]  Hédy Attouch,et al.  On the convergence of the proximal algorithm for nonsmooth functions involving analytic features , 2008, Math. Program..

[25]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[26]  Adrian S. Lewis,et al.  The [barred L]ojasiewicz Inequality for Nonsmooth Subanalytic Functions with Applications to Subgradient Dynamical Systems , 2006, SIAM J. Optim..

[27]  Amir Beck,et al.  First-Order Methods in Optimization , 2017 .

[28]  Marc Teboulle,et al.  Nonconvex Lagrangian-Based Optimization: Monitoring Schemes and Global Convergence , 2018, Math. Oper. Res..

[29]  Wotao Yin,et al.  Global Convergence of ADMM in Nonconvex Nonsmooth Optimization , 2015, Journal of Scientific Computing.

[30]  Radu Ioan Bot,et al.  An Inertial Tseng’s Type Proximal Algorithm for Nonsmooth and Nonconvex Optimization Problems , 2014, J. Optim. Theory Appl..

[31]  Xiaojun Chen,et al.  Alternating Direction Method of Multipliers for a Class of Nonconvex and Nonsmooth Problems with Applications to Background/Foreground Extraction , 2015, SIAM J. Imaging Sci..