Iterative ℓ1 minimization for non-convex compressed sensing

An algorithmic framework, based on the difference of convex functions algorithm, is proposed for minimizing a class of concave sparse metrics for compressed sensing problems. The resulting algorithm iterates a sequence of l1 minimization problems. An exact sparse recovery theory is established to show that the proposed framework always improves on the basis pursuit (l1 minimization) and inherits robustness from it. Numerical examples on success rates of sparse solution recovery illustrate further that, unlike most existing non-convex compressed sensing solvers in the literature, our method always out-performs basis pursuit, no matter how ill-conditioned the measurement matrix is. Moreover, the iterative l1 algorithms lead by a wide margin the state-of-the-art algorithms on l1/2 and logarithimic minimizations in the strongly coherent (highly ill-conditioned) regime, despite the same objective functions.

[1]  D. Donoho.  Superresolution via sparsity constraints , 1992 .

[2]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[3]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[4]  T. P. Dinh,et al.  Convex analysis approach to d.c. programming: Theory, Algorithm and Applications , 1997 .

[5]  W. Gander,et al.  A D.C. OPTIMIZATION ALGORITHM FOR SOLVING THE TRUST-REGION SUBPROBLEM∗ , 1998 .

[6]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[7]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[8]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[9]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[10]  J. Tropp,et al.  SIGNAL RECOVERY FROM PARTIAL INFORMATION VIA ORTHOGONAL MATCHING PURSUIT , 2005 .

[11]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[12]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[13]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[14]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[15]  R. Chartrand,et al.  Restricted isometry properties and nonconvex compressive sensing , 2007 .

[16]  Yin Zhang,et al.  Fixed-Point Continuation for l1-Minimization: Methodology and Convergence , 2008, SIAM J. Optim..

[17]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[18]  Wotao Yin,et al.  Bregman Iterative Algorithms for (cid:2) 1 -Minimization with Applications to Compressed Sensing ∗ , 2008 .

[19]  Mike E. Davies,et al.  Iterative Hard Thresholding for Compressed Sensing , 2008, ArXiv.

[20]  Tong Zhang,et al.  Multi-stage Convex Relaxation for Learning with Sparse Regularization , 2008, NIPS.

[21]  Wotao Yin,et al.  Iteratively reweighted algorithms for compressive sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[22]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[23]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[24]  Rick Chartrand,et al.  Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[25]  H. Rauhut.  Compressive Sensing and Structured Random Matrices , 2009 .

[26]  Jinchi Lv,et al.  A unified approach to model selection and sparse recovery using regularized least squares , 2009, 0905.3573.

[27]  Stéphane Canu,et al.  Recovering Sparse Signals With a Certain Family of Nonconvex Penalties and DC Programming , 2009, IEEE Transactions on Signal Processing.

[28]  Wotao Yin,et al.  Sparse Signal Reconstruction via Iterative Support Detection , 2009, SIAM J. Imaging Sci..

[29]  Ernie Esser,et al.  Applications of Lagrangian-Based Alternating Direction Methods and Connections to Split Bregman , 2009 .

[30]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[31]  Junfeng Yang,et al.  Alternating Direction Algorithms for 1-Problems in Compressive Sensing , 2009, SIAM J. Sci. Comput..

[32]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[33]  Emmanuel J. Candès,et al.  Towards a Mathematical Theory of Super‐resolution , 2012, ArXiv.

[34]  Wenjing Liao,et al.  Coherence Pattern-Guided Compressive Sensing with Unresolved Grids , 2011, SIAM J. Imaging Sci..

[35]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[36]  Jack Xin,et al.  A Method for Finding Structured Sparse Solutions to Nonnegative Least Squares Problems with Applications , 2013, SIAM J. Imaging Sci..

[37]  Wotao Yin,et al.  Improved Iteratively Reweighted Least Squares for Unconstrained Smoothed 퓁q Minimization , 2013, SIAM J. Numer. Anal..

[38]  Jack Xin,et al.  Ratio and difference of $l_1$ and $l_2$ norms and sparse representation with coherent dictionaries , 2014, Commun. Inf. Syst..

[39]  Jack Xin,et al.  Computing Sparse Representation in a Highly Coherent Dictionary Based on Difference of $$L_1$$L1 and $$L_2$$L2 , 2015, J. Sci. Comput..

[40]  Jack Xin,et al.  Minimization of ℓ1-2 for Compressed Sensing , 2015, SIAM J. Sci. Comput..

[41]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[42]  Jack Xin,et al.  Point Source Super-resolution Via Non-convex $$L_1$$L1 Based Methods , 2016, J. Sci. Comput..

[43]  Shenglong Zhou,et al.  A null-space-based weighted l1 minimization approach to compressed sensing , 2016 .