On the Convergence of the Iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm”

We discuss here the convergence of the iterates of the “Fast Iterative Shrinkage/Thresholding Algorithm,” which is an algorithm proposed by Beck and Teboulle for minimizing the sum of two convex, lower-semicontinuous, and proper functions (defined in a Euclidean or Hilbert space), such that one is differentiable with Lipschitz gradient, and the proximity operator of the second is easy to compute. It builds a sequence of iterates for which the objective is controlled, up to a (nearly optimal) constant, by the inverse of the square of the iteration number. However, the convergence of the iterates themselves is not known. We show here that with a small modification, we can ensure the same upper bound for the decay of the energy, as well as the convergence of the iterates to a minimizer.

[1]  Patrick L. Combettes,et al.  Proximal Splitting Methods in Signal Processing , 2009, Fixed-Point Algorithms for Inverse Problems in Science and Engineering.

[2]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[3]  Dimitri P. Bertsekas,et al.  On the Douglas—Rachford splitting method and the proximal point algorithm for maximal monotone operators , 1992, Math. Program..

[4]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[5]  John Darzentas,et al.  Problem Complexity and Method Efficiency in Optimization , 1983 .

[6]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[7]  Dirk A. Lorenz,et al.  An Inertial Forward-Backward Algorithm for Monotone Inclusions , 2014, Journal of Mathematical Imaging and Vision.

[8]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[9]  Yurii Nesterov,et al.  Smooth minimization of non-smooth functions , 2005, Math. Program..

[10]  Heinz H. Bauschke,et al.  Convex Analysis and Monotone Operator Theory in Hilbert Spaces , 2011, CMS Books in Mathematics.

[11]  Osman Güler,et al.  New Proximal Point Algorithms for Convex Minimization , 1992, SIAM J. Optim..

[12]  A. Moudafi,et al.  Convergence of a splitting inertial proximal method for monotone operators , 2003 .

[13]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[14]  P. Lions,et al.  Splitting Algorithms for the Sum of Two Nonlinear Operators , 1979 .

[15]  H. Attouch,et al.  An Inertial Proximal Method for Maximal Monotone Operators via Discretization of a Nonlinear Oscillator with Damping , 2001 .