A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration

Iterative shrinkage/thresholding (1ST) algorithms have been recently proposed to handle a class of convex unconstrained optimization problems arising in image restoration and other linear inverse problems. This class of problems results from combining a linear observation model with a nonquadratic regularizer (e.g., total variation or wavelet-based regularization). It happens that the convergence rate of these 1ST algorithms depends heavily on the linear observation operator, becoming very slow when this operator is ill-conditioned or ill-posed. In this paper, we introduce two-step 1ST (TwIST) algorithms, exhibiting much faster convergence rate than 1ST for ill-conditioned problems. For a vast class of nonquadratic convex regularizers (lscrP norms, some Besov norms, and total variation), we show that TwIST converges to a minimizer of the objective function, for a given range of values of its parameters. For noninvertible observation operators, we introduce a monotonic version of TwIST (MTwIST); although the convergence proof does not apply to this scenario, we give experimental evidence that MTwIST exhibits similar speed gains over IST. The effectiveness of the new methods are experimentally confirmed on problems of image deconvolution and of restoration with missing samples.

[1]  F. Clarke.  Optimization And Nonsmooth Analysis , 1983 .

[2]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[3]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[4]  P. Gács,et al.  Algorithms , 1992 .

[5]  D. M. Titterington,et al.  On some Bayesian/regularization methods for image restoration , 1995, IEEE Trans. Image Process..

[6]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[7]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[8]  P. Lions,et al.  Image recovery via total variation minimization and related problems , 1997 .

[9]  Aggelos K. Katsaggelos,et al.  Digital image restoration , 2012, IEEE Signal Process. Mag..

[10]  Antonin Chambolle,et al.  Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage , 1998, IEEE Trans. Image Process..

[11]  Yves Grandvalet,et al.  Outcomes of the Equivalence of Adaptive Ridge with Least Absolute Shrinkage , 1998, NIPS.

[12]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[13]  Mario Bertero,et al.  Introduction to Inverse Problems in Imaging , 1998 .

[14]  Wolfgang Osten,et al.  Introduction to Inverse Problems in Imaging , 1999 .

[15]  Pierre Moulin,et al.  Analysis of Multiresolution Image Denoising Schemes Using Generalized Gaussian and Complexity Priors , 1999, IEEE Trans. Inf. Theory.

[16]  C. E. M. Pearce,et al.  Some New Bounds for Singular Values and Eigenvalues of Matrix Products , 2000, Ann. Oper. Res..

[17]  L. Qi,et al.  Solving variational inequality problems via smoothing-nonsmooth reformulations , 2001 .

[18]  R. Nowak,et al.  Fast wavelet-based image deconvolution using the EM algorithm , 2001, Conference Record of Thirty-Fifth Asilomar Conference on Signals, Systems and Computers (Cat.No.01CH37256).

[19]  Z. Cao.  On the convergence of nonstationary iterative methods for symmetric positive (semi)definite systems , 2001 .

[20]  Nick G. Kingsbury,et al.  Image deconvolution using hidden Markov tree modeling of complex wavelet packets , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[21]  C. Zălinescu.  Convex analysis in general vector spaces , 2002 .

[22]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[23]  Mário A. T. Figueiredo.  Adaptive Sparseness for Supervised Learning , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[25]  E. Candès,et al.  Astronomical image representation by the curvelet transform , 2003, Astronomy & Astrophysics.

[26]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[27]  Fionn Murtagh,et al.  Fast communication , 2002 .

[28]  D. Lorenz.  Variational Denoising in Besov Spaces and Interpolation of Hard and Soft Wavelet Shrinkage , 2003 .

[29]  José M. Bioucas-Dias,et al.  Fast GEM wavelet-based image deconvolution algorithm , 2003, ICIP.

[30]  Antonin Chambolle,et al.  A l1-Unified Variational Framework for Image Restoration , 2004, ECCV.

[31]  Andy M. Yip,et al.  Recent Developments in Total Variation Image Restoration , 2004 .

[32]  D. Hunter,et al.  A Tutorial on MM Algorithms , 2004 .

[33]  Richard G. Baraniuk,et al.  ForWaRD: Fourier-wavelet regularized deconvolution for ill-conditioned systems , 2004, IEEE Transactions on Signal Processing.

[34]  Dirk A. Lorenz,et al.  Wavelet shrinkage in signal & image processing: an investigation of relations and equivalences , 2004 .

[35]  M. Nikolova.  An Algorithm for Total Variation Minimization and Applications , 2004 .

[36]  Jérôme Darbon,et al.  A Fast and Exact Algorithm for Total Variation Minimization , 2005, IbPRIA.

[37]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[38]  Robert D. Nowak,et al.  A bound optimization approach to wavelet-based image deconvolution , 2005, IEEE International Conference on Image Processing 2005.

[39]  J. Theys.  Joint spectral radius : theory and approximations/ , 2005 .

[40]  Antonin Chambolle,et al.  Total Variation Minimization and a Class of Binary MRF Models , 2005, EMMCVPR.

[41]  José M. Bioucas-Dias,et al.  Bayesian wavelet-based image deconvolution: a GEM algorithm exploiting a class of heavy-tailed priors , 2006, IEEE Transactions on Image Processing.

[42]  Michael Elad,et al.  Image Denoising with Shrinkage and Redundant Representations , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[43]  José M. Bioucas-Dias,et al.  Total Variation-Based Image Deconvolution: a Majorization-Minimization Approach , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[44]  Michael Elad,et al.  Why Simple Shrinkage Is Still Relevant for Redundant Representations? , 2006, IEEE Transactions on Information Theory.

[45]  Javier Portilla,et al.  Deblurring-by-Denoising using Spatially Adaptive Gaussian Scale Mixtures in Overcomplete Pyramids , 2006, 2006 International Conference on Image Processing.

[46]  José M. Bioucas-Dias,et al.  Adaptive total variation image deconvolution: A majorization-minimization approach , 2006, 2006 14th European Signal Processing Conference.

[47]  Robert D. Nowak,et al.  On Total Variation Denoising: A New Majorization-Minimization Algorithm and an Experimental Comparisonwith Wavalet Denoising , 2006, 2006 International Conference on Image Processing.

[48]  Patrick L. Combettes,et al.  Iterative image deconvolution using overcomplete representations , 2006, 2006 14th European Signal Processing Conference.

[49]  Robert D. Nowak,et al.  Majorization–Minimization Algorithms for Wavelet-Based Image Restoration , 2007, IEEE Transactions on Image Processing.

[50]  José M. Bioucas-Dias,et al.  Two-Step Algorithms for Linear Inverse Problems with Non-Quadratic Regularization , 2007, 2007 IEEE International Conference on Image Processing.