Accelerating gradient projection methods for ℓ1-constrained signal recovery by steplength selection rules

Abstract We propose a new gradient projection algorithm that compares favorably with the fastest algorithms available to date for l 1 -constrained sparse recovery from noisy data, both in the compressed sensing and inverse problem frameworks. The method exploits a line-search along the feasible direction and an adaptive steplength selection based on recent strategies for the alternation of the well-known Barzilai–Borwein rules. The convergence of the proposed approach is discussed and a computational study on both well conditioned and ill-conditioned problems is carried out for performance evaluations in comparison with five other algorithms proposed in the literature.

[1]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[2]  Roger Fletcher,et al.  On the asymptotic behaviour of some new gradient methods , 2005, Math. Program..

[3]  L. Zanni,et al.  A scaled gradient projection method for constrained image deblurring , 2008 .

[4]  L. Grippo,et al.  A nonmonotone line search technique for Newton's method , 1986 .

[5]  L. Zanni,et al.  New adaptive stepsize selections in gradient methods , 2008 .

[6]  I. Loris.  On the performance of algorithms for the minimization of ℓ1-penalized functionals , 2007, 0710.4082.

[7]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[8]  Roger Fletcher,et al.  On the Barzilai-Borwein Method , 2005 .

[9]  W. Hager,et al.  The cyclic Barzilai-–Borwein method for unconstrained optimization , 2006 .

[10]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[11]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[12]  J. M. Martínez,et al.  Inexact spectral projected gradient methods on convex sets , 2003 .

[13]  J. Borwein,et al.  Two-Point Step Size Gradient Methods , 1988 .

[14]  Luca Zanni,et al.  Gradient projection methods for quadratic programs and applications in training support vector machines , 2005, Optim. Methods Softw..

[15]  I. Daubechies,et al.  Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints , 2007, 0706.4297.

[16]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[17]  M. R. Osborne,et al.  A new approach to variable selection in least squares problems , 2000 .

[18]  Roger Fletcher,et al.  New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds , 2006, Math. Program..

[19]  Bin Zhou,et al.  Gradient Methods with Adaptive Step-Sizes , 2006, Comput. Optim. Appl..

[20]  Luca Zanni,et al.  An Improved Gradient Projection-based Decomposition Technique for Support Vector Machines , 2006, Comput. Manag. Sci..

[21]  I. Daubechies,et al.  Tomographic inversion using L1-norm regularization of wavelet coefficients , 2006, physics/0608094.

[22]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[23]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[24]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[25]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[26]  Mário A. T. Figueiredo,et al.  Gradient Projection for Sparse Reconstruction: Application to Compressed Sensing and Other Inverse Problems , 2007, IEEE Journal of Selected Topics in Signal Processing.

[27]  Dimitri P. Bertsekas,et al.  Nonlinear Programming , 1997 .

[28]  Patrick L. Combettes,et al.  Signal Recovery by Proximal Forward-Backward Splitting , 2005, Multiscale Model. Simul..

[29]  Robert D. Nowak,et al.  An EM algorithm for wavelet-based image restoration , 2003, IEEE Trans. Image Process..

[30]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[31]  J. M. Martínez,et al.  Gradient Method with Retards and Generalizations , 1998 .