On the rate of convergence of the proximal alternating linearized minimization algorithm for convex problems

We analyze the proximal alternating linearized minimization algorithm (PALM) for solving non-smooth convex minimization problems where the objective function is a sum of a smooth convex function and block separable non-smooth extended real-valued convex functions. We prove a global non-asymptotic sublinear rate of convergence for PALM. When the number of blocks is two, and the smooth coupling function is quadratic we present a fast version of PALM which is proven to share a global sublinear rate efficiency estimate improved by a squared root factor. Some numerical examples illustrate the potential benefits of the proposed schemes.

[1]  Luigi Grippo,et al.  On the convergence of the block nonlinear Gauss-Seidel method under convex constraints , 2000, Oper. Res. Lett..

[2]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[3]  A. Chambolle,et al.  A remark on accelerated block coordinate descent for computing the proximity operators of a sum of convex functions , 2015 .

[4]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[5]  A. Auslender.  Méthodes numériques pour la décomposition et la minimisation de fonctions non différentiables , 1971 .

[6]  A. Auslender.  Optimisation : méthodes numériques , 1976 .

[7]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[8]  ITERATIVE SOLUTION OF NONLINEAR EQUATIONS OF HAMMERSTEIN TYPE , 2003 .

[9]  Amir Beck,et al.  On the Convergence of Block Coordinate Descent Type Methods , 2013, SIAM J. Optim..

[10]  Yonina C. Eldar,et al.  Convex Optimization in Signal Processing and Communications , 2009 .

[11]  Stephen J. Wright,et al.  Optimization for Machine Learning , 2013 .

[12]  John N. Tsitsiklis,et al.  Parallel and distributed computation , 1989 .

[13]  Yurii Nesterov,et al.  Efficiency of Coordinate Descent Methods on Huge-Scale Optimization Problems , 2012, SIAM J. Optim..

[14]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[15]  J. Moreau.  Proximité et dualité dans un espace hilbertien , 1965 .

[16]  Marc Teboulle,et al.  Gradient-based algorithms with applications to signal-recovery problems , 2010, Convex Optimization in Signal Processing and Communications.

[17]  M. J. D. Powell,et al.  On search directions for minimization algorithms , 1973, Math. Program..

[18]  P. Tseng.  Convergence of a Block Coordinate Descent Method for Nondifferentiable Minimization , 2001 .

[19]  Amir Beck,et al.  On the Convergence of Alternating Minimization for Convex Programming with Applications to Iteratively Reweighted Least Squares and Decomposition Schemes , 2015, SIAM J. Optim..