Generalized Arnoldi-Tikhonov Method for Sparse Reconstruction

This paper introduces two new algorithms, belonging to the class of Arnoldi--Tikhonov regularization methods, which are particularly appropriate for sparse reconstruction. The main idea is to consider suitable adaptively defined regularization matrices that allow the usual 2-norm regularization term to approximate a more general regularization term expressed in the $p$-norm, $p\geq 1$. The regularization matrix can be updated both at each step and after some iterations have been performed, leading to two different approaches: the first one is based on the idea of the iteratively reweighted least squares method and can be obtained considering flexible Krylov subspaces; the second one is based on restarting the Arnoldi algorithm. Numerical examples are given in order to show the effectiveness of these new methods, and comparisons with some other already existing algorithms are made.

[1]  Lea Fleischer,et al.  Regularization of Inverse Problems , 1996 .

[2]  J. Nagy,et al.  Enforcing nonnegativity in image reconstruction algorithms , 2000, SPIE Optics + Photonics.

[3]  Dianne P. O'Leary,et al.  Restoring Images Degraded by Spatially Variant Blur , 1998, SIAM J. Sci. Comput..

[4]  D. O’Leary,et al.  A Bidiagonalization-Regularization Procedure for Large Scale Discretizations of Ill-Posed Problems , 1981 .

[5]  Paolo Novati,et al.  Automatic parameter setting for Arnoldi-Tikhonov methods , 2014, J. Comput. Appl. Math..

[6]  Michiel E. Hochstenbach,et al.  An iterative method for Tikhonov regularization with a general linear regularization operator , 2010 .

[7]  Å. Björck.  A bidiagonalization algorithm for solving large and sparse ill-posed systems of linear equations , 1988 .

[8]  José M. Bioucas-Dias,et al.  A New TwIST: Two-Step Iterative Shrinkage/Thresholding Algorithms for Image Restoration , 2007, IEEE Transactions on Image Processing.

[9]  Emmanuel J. Candès,et al.  NESTA: A Fast and Accurate First-Order Method for Sparse Recovery , 2009, SIAM J. Imaging Sci..

[10]  Valeria Simoncini,et al.  Flexible Inner-Outer Krylov Subspace Methods , 2002, SIAM J. Numer. Anal..

[11]  Per Christian Hansen,et al.  Smoothing-Norm Preconditioning for Regularizing Minimum-Residual Methods , 2006, SIAM J. Matrix Anal. Appl..

[12]  B. Wohlberg,et al.  An Iteratively Reweighted Norm Algorithm for Total Variation Regularization , 2006, 2006 Fortieth Asilomar Conference on Signals, Systems and Computers.

[13]  Stephen P. Boyd,et al.  An Interior-Point Method for Large-Scale $\ell_1$-Regularized Least Squares , 2007, IEEE Journal of Selected Topics in Signal Processing.

[14]  R. Larsen.  Lanczos Bidiagonalization With Partial Reorthogonalization , 1998 .

[15]  Yousef Saad,et al.  A Flexible Inner-Outer Preconditioned GMRES Algorithm , 1993, SIAM J. Sci. Comput..

[16]  Åke Björck,et al.  An implicit shift bidiagonalization algorithm for ill-posed systems , 1994 .

[17]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[18]  Lothar Reichel,et al.  Tikhonov regularization based on generalized Krylov subspace methods , 2012 .

[19]  J. Nagy,et al.  A weighted-GCV method for Lanczos-hybrid regularization. , 2007 .

[20]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[21]  Brendt Wohlberg,et al.  An efficient algorithm for sparse representations with lp data fidelity term , 2008 .

[22]  Lothar Reichel,et al.  Invertible smoothing preconditioners for linear discrete ill-posed problems , 2005 .

[23]  D. Calvetti,et al.  Tikhonov regularization and the L-curve for large discrete ill-posed problems , 2000 .

[24]  Samuli Siltanen,et al.  Linear and Nonlinear Inverse Problems with Practical Applications , 2012, Computational science and engineering.

[25]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[26]  Brendt Wohlberg,et al.  UPRE method for total variation parameter selection , 2010, Signal Process..

[27]  C. Vogel.  Computational Methods for Inverse Problems , 1987 .

[28]  Jun-Feng Yin,et al.  GMRES Methods for Least Squares Problems , 2010, SIAM J. Matrix Anal. Appl..

[29]  D. Calvetti,et al.  Tikhonov Regularization of Large Linear Problems , 2003 .

[30]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[31]  Misha Elena Kilmer,et al.  A Projection-Based Approach to General-Form Tikhonov Regularization , 2007, SIAM J. Sci. Comput..

[32]  Lothar Reichel,et al.  Arnoldi-Tikhonov regularization methods , 2009 .

[33]  Lothar Reichel,et al.  Non-negativity and iterative methods for ill-posed problems , 2004 .

[34]  James G. Nagy,et al.  Iterative Methods for Image Deblurring: A Matlab Object-Oriented Approach , 2004, Numerical Algorithms.

[35]  P. Hansen.  Discrete Inverse Problems: Insight and Algorithms , 2010 .

[36]  Brendt Wohlberg,et al.  An Iteratively Reweighted Norm Algorithm for Minimization of Total Variation Functionals , 2007, IEEE Signal Processing Letters.

[37]  D. Hunter,et al.  A Tutorial on MM Algorithms , 2004 .

[38]  José M. Bioucas-Dias,et al.  Adaptive total variation image deblurring: A majorization-minimization approach , 2009, Signal Process..

[39]  Martin Hanke,et al.  On Lanczos Based Methods for the Regularization of Discrete Ill-Posed Problems , 2001 .

[40]  Valeria Simoncini,et al.  Recent computational developments in Krylov subspace methods for linear systems , 2007, Numer. Linear Algebra Appl..

[41]  Misha Elena Kilmer,et al.  Choosing Regularization Parameters in Iterative Methods for Ill-Posed Problems , 2000, SIAM J. Matrix Anal. Appl..

[42]  Francoise Preteux,et al.  Mathematical Modeling, Estimation, and Imaging , 2000 .

[43]  Yousef Saad,et al.  Iterative methods for sparse linear systems , 2003 .