Accelerated dual gradient-based methods for total variation image denoising/deblurring problems

We study accelerated dual gradient-based methods for image denoising/deblurring problems based on the total variation (TV) model. For the TV-based denoising problem, combining the dual approach and Nesterov's fast gradient projection (FGP) method has been found effective. The corresponding denoising method minimizes the dual function with FGP's optimal rate O(1/k2) where k denotes the number of iterations, and guarantees a rate O(1/k) for the primal function decrease. Considering that the dual projected gradient decrease is closely related to the primal function decrease, this paper proposes new accelerated gradient projection methods that decrease the projected gradient norm with a fast rate O(1/k1.5) and that are as efficient as FGP. The proposed approach also decreases the primal function with a faster rate O(1/k1.5). We provide preliminary results on image denoising/deblurring problems with a TV regularizer, where the fast and efficient denoising solver is iteratively used for solving a deblurring problem as the inner proximal update of a fast iterative shrinkage/thresholding algorithm (FISTA).

[1]  Volkan Cevher,et al.  Convex Optimization for Big Data: Scalable, randomized, and parallel algorithms for big data analytics , 2014, IEEE Signal Processing Magazine.

[2]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[3]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[4]  Jeffrey A. Fessler,et al.  On the Convergence Analysis of the Optimized Gradient Method , 2015, J. Optim. Theory Appl..

[5]  Yurii Nesterov,et al.  Introductory Lectures on Convex Optimization - A Basic Course , 2014, Applied Optimization.

[6]  Jeffrey A. Fessler,et al.  Fast dual proximal gradient algorithms with rate $O(1/k^{1.5})$ for convex minimization , 2016 .

[7]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[8]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[9]  Jeffrey A. Fessler,et al.  Generalizing the Optimized Gradient Method for Smooth Convex Minimization , 2016, SIAM J. Optim..

[10]  Zeyuan Allen-Zhu,et al.  How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD , 2018, NeurIPS.

[11]  Donghwan Kim,et al.  Optimized first-order methods for smooth convex minimization , 2014, Mathematical Programming.

[12]  Marc Teboulle,et al.  Fast Gradient-Based Algorithms for Constrained Total Variation Image Denoising and Deblurring Problems , 2009, IEEE Transactions on Image Processing.

[13]  Ulugbek Kamilov,et al.  A Parallel Proximal Algorithm for Anisotropic Total Variation Minimization , 2017, IEEE Transactions on Image Processing.

[14]  Jeffrey A. Fessler,et al.  Another Look at the Fast Iterative Shrinkage/Thresholding Algorithm (FISTA) , 2016, SIAM J. Optim..

[15]  Marc Teboulle,et al.  A fast dual proximal gradient algorithm for convex minimization and applications , 2014, Oper. Res. Lett..

[16]  Antonin Chambolle,et al.  Total Variation Minimization and a Class of Binary MRF Models , 2005, EMMCVPR.