Compressed sensing recovery via nonconvex shrinkage penalties

The $\ell^0$ minimization of compressed sensing is often relaxed to $\ell^1$, which yields easy computation using the shrinkage mapping known as soft thresholding, and can be shown to recover the original solution under certain hypotheses. Recent work has derived a general class of shrinkages and associated nonconvex penalties that better approximate the original $\ell^0$ penalty and empirically can recover the original solution from fewer measurements. We specifically examine p-shrinkage and firm thresholding. In this work, we prove that given data and a measurement matrix from a broad class of matrices, one can choose parameters for these classes of shrinkages to guarantee exact recovery of the sparsest solution. We further prove convergence of the algorithm iterative p-shrinkage (IPS) for solving one such relaxed problem.

[1]  Rayan Saab,et al.  Stable sparse approximations via nonconvex optimization , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[2]  Michael Elad,et al.  A generalized uncertainty principle and sparse representation in pairs of bases , 2002, IEEE Trans. Inf. Theory.

[3]  Andrea Montanari,et al.  Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising , 2011, IEEE Transactions on Information Theory.

[4]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[5]  Rick Chartrand,et al.  Shrinkage mappings and their induced penalty functions , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[6]  Rui Wu,et al.  The Improved Bounds of Restricted Isometry Constant for Recovery via $\ell_{p}$ -Minimization , 2013, IEEE Transactions on Information Theory.

[7]  Bin Dong,et al.  An Efficient Algorithm for ℓ0 Minimization in Wavelet Frame Based Image Restoration , 2013, J. Sci. Comput..

[8]  S. Foucart,et al.  Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0 , 2009 .

[9]  A. Bruce,et al.  WAVESHRINK WITH FIRM SHRINKAGE , 1997 .

[10]  R. Chartrand,et al.  Restricted isometry properties and nonconvex compressive sensing , 2007 .

[11]  Y. Ye,et al.  Lower Bound Theory of Nonzero Entries in Solutions of ℓ2-ℓp Minimization , 2010, SIAM J. Sci. Comput..

[12]  Rick Chartrand,et al.  Nonconvex Splitting for Regularized Low-Rank + Sparse Decomposition , 2012, IEEE Transactions on Signal Processing.

[13]  S. Muthukrishnan,et al.  Data streams: algorithms and applications , 2005, SODA '03.

[14]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[15]  A. Majumdar,et al.  An algorithm for sparse MRI reconstruction by Schatten p-norm minimization. , 2011, Magnetic resonance imaging.

[16]  Shun-ichi Amari,et al.  ℓp-constrained least squares (0 < p < 1) and its critical path , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[17]  B. Mercier,et al.  A dual algorithm for the solution of nonlinear variational problems via finite element approximation , 1976 .

[18]  L. Schumaker,et al.  Approximation theory XIII : San Antonio 2010 , 2012 .

[19]  S. Foucart A note on guaranteed sparse recovery via ℓ1-minimization , 2010 .

[20]  S. Amari,et al.  $\ell _{p}$ -Regularized Least Squares $(0 and Critical Path , 2016, IEEE Transactions on Information Theory.

[21]  Davies Rémi Gribonval Restricted Isometry Constants Where Lp Sparse Recovery Can Fail for 0 , 2008 .

[22]  Akram Aldroubi,et al.  Stability and robustness of lq minimization using null space property , 2011 .

[23]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[24]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[25]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[26]  Rick Chartrand,et al.  Exact Reconstruction of Sparse Signals via Nonconvex Minimization , 2007, IEEE Signal Processing Letters.

[27]  Brendt Wohlberg,et al.  A nonconvex ADMM algorithm for group sparsity with sparse groups , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[28]  Qiyu Sun,et al.  Recovery of sparsest signals via ℓq-minimization , 2010, ArXiv.

[29]  Rick Chartrand Generalized shrinkage and penalty functions , 2013, 2013 IEEE Global Conference on Signal and Information Processing.

[30]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[31]  R. Glowinski,et al.  Sur l'approximation, par éléments finis d'ordre un, et la résolution, par pénalisation-dualité d'une classe de problèmes de Dirichlet non linéaires , 1975 .

[32]  Anru Zhang,et al.  Sharp RIP bound for sparse signal and low-rank matrix recovery , 2013 .

[33]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[34]  Emil Y. Sidky,et al.  Nonconvex compressive sensing for X-ray CT: An algorithm comparison , 2013, 2013 Asilomar Conference on Signals, Systems and Computers.

[35]  I. Daubechies,et al.  An iterative thresholding algorithm for linear inverse problems with a sparsity constraint , 2003, math/0307152.

[36]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[37]  Rick Chartrand,et al.  A new generalized thresholding algorithm for inverse problems with sparsity constraints , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[38]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[39]  Anestis Antoniadis,et al.  Wavelet methods in statistics: Some recent developments and their applications , 2007, 0712.0283.

[40]  S. Foucart Sparse Recovery Algorithms: Sufficient Conditions in Terms of RestrictedIsometry Constants , 2012 .

[41]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[42]  R. Gribonval,et al.  Highly sparse representations from dictionaries are unique and independent of the sparseness measure , 2007 .

[43]  Rémi Gribonval,et al.  Restricted Isometry Constants Where $\ell ^{p}$ Sparse Recovery Can Fail for $0≪ p \leq 1$ , 2009, IEEE Transactions on Information Theory.

[44]  Rick Chartrand,et al.  Nonconvex Compressed Sensing and Error Correction , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[45]  M. Lai,et al.  The null space property for sparse recovery from multiple measurement vectors , 2011 .

[46]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[47]  Qiyu Sun Sparse Approximation Property and Stable Recovery of Sparse Signals From Noisy Measurements , 2011, IEEE Transactions on Signal Processing.

[48]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[49]  Rick Chartrand Nonconvex compressive sensing and reconstruction of gradient-sparse images: Random vs. tomographic Fourier sampling , 2008, 2008 15th IEEE International Conference on Image Processing.

[50]  Rick Chartrand,et al.  Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[51]  Robert R. Meyer,et al.  Sufficient Conditions for the Convergence of Monotonic Mathematical Programming Algorithms , 1976, J. Comput. Syst. Sci..

[52]  Bastian Goldlücke,et al.  Variational Analysis , 2014, Computer Vision, A Reference Guide.