Domain decomposition methods for linear inverse problems with sparsity constraints

Quantities of interest appearing in concrete applications often possess sparse expansions with respect to a preassigned frame. Recently, there were introduced sparsity measures which are typically constructed on the basis of weighted l1 norms of frame coefficients. One can model the reconstruction of a sparse vector from noisy linear measurements as the minimization of the functional defined by the sum of the discrepancy with respect to the data and the weighted l1-norm of suitable frame coefficients. Thresholded Landweber iterations were proposed for the solution of the variational problem. Despite its simplicity which makes it very attractive to users, this algorithm converges slowly. In this paper, we investigate methods to accelerate significantly the convergence. We introduce and analyze sequential and parallel iterative algorithms based on alternating subspace corrections for the solution of the linear inverse problem with sparsity constraints. We prove their norm convergence to minimizers of the functional. We compare the computational cost and the behavior of these new algorithms with respect to the thresholded Landweber iterations.

[1]  Z. Opial Weak convergence of the sequence of successive approximations for nonexpansive mappings , 1967 .

[2]  I. Ekeland,et al.  Convex analysis and variational problems , 1976 .

[3]  J. Pasciak,et al.  Convergence estimates for product iterative methods with applications to domain decomposition , 1991 .

[4]  Jinchao Xu,et al.  Iterative Methods by Space Decomposition and Subspace Correction , 1992, SIAM Rev..

[5]  O. Widlund Domain Decomposition Algorithms , 1993 .

[6]  T. Chan,et al.  Domain decomposition algorithms , 1994, Acta Numerica.

[7]  Alfio Quarteroni,et al.  Domain Decomposition Methods for Partial Differential Equations , 1999 .

[8]  H. Engl,et al.  Regularization of Inverse Problems , 1996 .

[9]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[10]  P. Lions,et al.  Image recovery via total variation minimization and related problems , 1997 .

[11]  Antonin Chambolle,et al.  Nonlinear wavelet image processing: variational problems, compression, and noise removal through wavelet shrinkage , 1998, IEEE Trans. Image Process..

[12]  Zi-Cai Li,et al.  Schwarz Alternating Method , 1998 .

[13]  O. Christensen An introduction to frames and Riesz bases , 2002 .

[14]  Wolfgang Dahmen,et al.  Adaptive Wavelet Methods II—Beyond the Elliptic Case , 2002, Found. Comput. Math..

[15]  Jinchao Xu,et al.  The method of alternating projections and the method of subspace corrections in Hilbert space , 2002 .

[16]  Successive Subspace Correction method for Singular System of Equations , 2003 .

[17]  Wolfgang Dahmen,et al.  Adaptive Wavelet Schemes for Nonlinear Variational Problems , 2003, SIAM J. Numer. Anal..

[18]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[19]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[20]  E. Candès,et al.  Astronomical image representation by the curvelet transform , 2003, Astronomy & Astrophysics.

[21]  Heinz H. Bauschke,et al.  Accelerating the convergence of the method of alternating projections , 2003 .

[22]  Fionn Murtagh,et al.  Fast communication , 2002 .

[23]  Rob P. Stevenson,et al.  Adaptive Solution of Operator Equations Using Wavelet Frames , 2003, SIAM J. Numer. Anal..

[24]  Ingrid Daubechies,et al.  Variational image restoration by means of wavelets: simultaneous decomposition , 2005 .

[25]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[26]  Laurent D. Cohen,et al.  Auxiliary variables and two-step iterative algorithms in computer vision problems , 2004, Journal of Mathematical Imaging and Vision.

[27]  I. Daubechiesa,et al.  Variational image restoration by means of wavelets : Simultaneous decomposition , deblurring , and denoising , 2005 .

[28]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[29]  G. Teschke,et al.  Tikhonov replacement functionals for iteratively solving nonlinear operator equations , 2005 .

[30]  D. Donoho,et al.  Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA) , 2005 .

[31]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[32]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[33]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[34]  Daniel B. Szyld,et al.  Schwarz Iterations for Symmetric Positive Semidefinite Problems , 2006, SIAM J. Matrix Anal. Appl..

[35]  Ronny Ramlau,et al.  A Tikhonov-based projection iteration for nonlinear Ill-posed problems with sparsity constraints , 2006, Numerische Mathematik.

[36]  Massimo Fornasier,et al.  Restoration of Color Images by Vector Valued BV Functions and Variational Calculus , 2007, SIAM J. Appl. Math..

[37]  I. Daubechies,et al.  Tomographic inversion using L1-norm regularization of wavelet coefficients , 2006, physics/0608094.

[38]  G. Teschke Multi-frame representations in linear inverse problems with mixed multi-constraints , 2007 .

[39]  I. Daubechies,et al.  Iteratively solving linear inverse problems under general convex constraints , 2007 .

[40]  M. Fornasier,et al.  Adaptive Frame Methods for Elliptic Operator Equations: The Steepest Descent Approach , 2007 .

[41]  Massimo Fornasier,et al.  Adaptive frame methods for elliptic operator equations , 2007, Adv. Comput. Math..

[42]  Massimo Fornasier,et al.  Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints , 2008, SIAM J. Numer. Anal..

[43]  I. Daubechies,et al.  Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints , 2007, 0706.4297.

[44]  M. Fornasier,et al.  Adaptive iterative thresholding algorithms for magnetoencephalography (MEG) , 2008 .

[45]  J. Tropp On the conditioning of random subdictionaries , 2008 .

[46]  M. Fornasier,et al.  Iterative thresholding algorithms , 2008 .