Block-length dependent thresholds in block-sparse compressed sensing

One of the most basic problems in compressed sensing is solving an under-determined system of linear equations. Although this problem seems rather hard certain l1-optimization algorithm appears to be very successful in solving it. The recent work of [14, 28] rigorously proved (in a large dimensional and statistical context) that if the number of equations (measurements in the compressed sensing terminology) in the system is proportional to the length of the unknown vector then there is a sparsity (number of non-zero elements of the unknown vector) also proportional to the length of the unknown vector such that l1-optimization algorithm succeeds in solving the system. In more recent papers [78,81] we considered the setup of the so-called block-sparse unknown vectors. In a large dimensional and statistical context, we determined sharp lower bounds on the values of allowable sparsity for any given number (proportional to the length of the unknown vector) of equations such that an l2/l1-optimization algorithm succeeds in solving the system. The results established in [78, 81] assumed a fairly large block-length of the block-sparse vectors. In this paper we consider the block-length to be a parameter of the system. Consequently, we then establish sharp lower bounds on the values of the allowable block-sparsity as functions of the block-length.

[1]  H. Ruben.  On the geometrical moments of skew-regular simplices in hyperspherical space, with some applications in geometry and mathematical statistics , 1960 .

[2]  Martin J. Wainwright,et al.  Simultaneous Support Recovery in High Dimensions: Benefits and Perils of Block $\ell _{1}/\ell _{\infty} $-Regularization , 2009, IEEE Transactions on Information Theory.

[3]  J. K. Böröczky,et al.  Random projections of regular polytopes , 1999 .

[4]  Alex Samorodnitsky,et al.  Random weighting, asymptotic counting, and inverse isoperimetry , 2005, Electron. Colloquium Comput. Complex..

[5]  Weiyu Xu,et al.  Efficient Compressive Sensing with Deterministic Guarantees Using Expander Graphs , 2007, 2007 IEEE Information Theory Workshop.

[6]  D. Donoho,et al.  Thresholds for the Recovery of Sparse Solutions via L1 Minimization , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[7]  M. Wainwright,et al.  Simultaneous support recovery in high dimensions : Benefits and perils of block l 1 / l ∞-regularization , 2009 .

[8]  M. Stojnic,et al.  $\ell_{2}/\ell_{1}$ -Optimization in Block-Sparse Compressed Sensing and Its Strong Thresholds , 2010, IEEE Journal of Selected Topics in Signal Processing.

[9]  D. Donoho,et al.  Neighborliness of randomly projected simplices in high dimensions. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Rayan Saab,et al.  Stable sparse approximations via nonconvex optimization , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  G. Pisier.  Probabilistic methods in the geometry of Banach spaces , 1986 .

[12]  Martin Vetterli,et al.  Sampling and reconstruction of signals with finite rate of innovation in the presence of noise , 2005, IEEE Transactions on Signal Processing.

[13]  Rémi Gribonval,et al.  Restricted Isometry Constants Where $\ell ^{p}$ Sparse Recovery Can Fail for $0≪ p \leq 1$ , 2009, IEEE Transactions on Information Theory.

[14]  Martin J. Wainwright,et al.  Sharp thresholds for high-dimensional and noisy recovery of sparsity , 2006, ArXiv.

[15]  Mihailo Stojnic.  Explicit thresholds for approximately sparse compressed sensing via ℓ1-optimization , 2009, 2009 IEEE International Symposium on Information Theory.

[16]  Stéphane Chrétien,et al.  An Alternating $l_1$ Approach to the Compressed Sensing Problem , 2008, IEEE Signal Processing Letters.

[17]  David L. Donoho,et al.  Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[18]  B. Hassibi,et al.  On Recovery of Sparse Signals in Compressed DNA Microarrays , 2007, 2007 Conference Record of the Forty-First Asilomar Conference on Signals, Systems and Computers.

[19]  J. Romberg,et al.  Imaging via Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[20]  Yonina C. Eldar,et al.  Reduce and Boost: Recovering Arbitrary Sets of Jointly Sparse Vectors , 2008, IEEE Transactions on Signal Processing.

[21]  Jie Chen,et al.  Theoretical Results on Sparse Representations of Multiple-Measurement Vectors , 2006, IEEE Transactions on Signal Processing.

[22]  D. Donoho,et al.  Sparse nonnegative solution of underdetermined linear equations by linear programming. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Weiyu Xu,et al.  Compressed sensing of approximately sparse signals , 2008, 2008 IEEE International Symposium on Information Theory.

[24]  Joel A. Tropp,et al.  Algorithms for simultaneous sparse approximation. Part I: Greedy pursuit , 2006, Signal Process..

[25]  Weiyu Xu,et al.  Weighted ℓ1 minimization for sparse recovery with prior information , 2009, 2009 IEEE International Symposium on Information Theory.

[26]  Mihailo Stojnic,et al.  Strong thresholds for ℓ2/ℓ1-optimization in block-sparse compressed sensing , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[27]  Nathan Linial,et al.  How Neighborly Can a Centrally Symmetric Polytope Be? , 2006, Discret. Comput. Geom..

[28]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[29]  Arkadi Nemirovski,et al.  On sparse representation in pairs of bases , 2003, IEEE Trans. Inf. Theory.

[30]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[31]  Robert D. Nowak,et al.  Signal Reconstruction From Noisy Random Projections , 2006, IEEE Transactions on Information Theory.

[32]  Bhaskar D. Rao,et al.  Sparse solutions to linear inverse problems with multiple measurement vectors , 2005, IEEE Transactions on Signal Processing.

[33]  R. Vershynin,et al.  One sketch for all: fast algorithms for compressed sensing , 2007, STOC '07.

[34]  Rolf Schneider,et al.  Random projections of regular simplices , 1992, Discret. Comput. Geom..

[35]  E.J. Candes.  Compressive Sampling , 2022 .

[36]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[37]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[38]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.

[39]  R. Gribonval,et al.  Highly sparse representations from dictionaries are unique and independent of the sparseness measure , 2007 .

[40]  Babak Hassibi,et al.  Recovering Sparse Signals Using Sparse Measurement Matrices in Compressed DNA Microarrays , 2008, IEEE Journal of Selected Topics in Signal Processing.

[41]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[42]  M. Rudelson,et al.  Geometric approach to error-correcting codes and reconstruction of signals , 2005, math/0502299.

[43]  Yonina C. Eldar,et al.  Block-sparsity: Coherence and efficient recovery , 2008, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  M. Rudelson,et al.  On sparse reconstruction from Fourier and Gaussian measurements , 2008 .

[45]  Michael P. Friedlander,et al.  Theoretical and Empirical Results for Recovery From Multiple Measurements , 2009, IEEE Transactions on Information Theory.

[46]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[47]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[48]  Babak Hassibi,et al.  Explicit measurements with almost optimal thresholds for compressed sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[49]  S. Stigler.  The Asymptotic Distribution of the Trimmed Mean , 1973 .

[50]  I. Ibragimov,et al.  Norms of Gaussian sample functions , 1976 .

[51]  Venkatesh Saligrama,et al.  Thresholded Basis Pursuit: Quantizing Linear Programming Solutions for Optimal Support Recovery and Approximation in Compressed Sensing , 2008, ArXiv.

[52]  Guillermo Sapiro,et al.  Sparse representations for image classification: learning discriminative and reconstructive non-parametric dictionaries , 2008 .

[53]  Mike E. Davies,et al.  Sampling Theorems for Signals From the Union of Finite-Dimensional Linear Subspaces , 2009, IEEE Transactions on Information Theory.

[54]  Yin Zhang Caam.  When is missing data recoverable ? , 2006 .

[55]  Mihailo Stojnic,et al.  Various thresholds for ℓ1-optimization in compressed sensing , 2009, ArXiv.

[56]  Piotr Indyk,et al.  Sparse Recovery Using Sparse Random Matrices , 2010, LATIN.

[57]  C EldarYonina,et al.  Robust recovery of signals from a structured union of subspaces , 2009 .

[58]  R. DeVore,et al.  Compressed sensing and best k-term approximation , 2008 .

[59]  Yonina C. Eldar,et al.  Block-Sparse Signals: Uncertainty Relations and Efficient Recovery , 2009, IEEE Transactions on Signal Processing.

[60]  John Wright,et al.  Dense Error Correction Via $\ell^1$-Minimization , 2010, IEEE Transactions on Information Theory.

[61]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[62]  David L. Donoho,et al.  High-Dimensional Centrally Symmetric Polytopes with Neighborliness Proportional to Dimension , 2006, Discret. Comput. Geom..

[63]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[64]  Weiyu Xu,et al.  Breaking through the thresholds: an analysis for iterative reweighted ℓ1 minimization via the Grassmann angle framework , 2009, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[65]  Zihan Zhou,et al.  Separation of a subspace-sparse signal: Algorithms and conditions , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[66]  Rémi Gribonval,et al.  On the Strong Uniqueness of Highly Sparse Representations from Redundant Dictionaries , 2004, ICA.

[67]  David L. Donoho,et al.  Neighborly Polytopes And Sparse Solution Of Underdetermined Linear Equations , 2005 .

[68]  Y. Gordon.  On Milman's inequality and random subspaces which escape through a mesh in ℝ n , 1988 .

[69]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[70]  Graham Cormode,et al.  Combinatorial Algorithms for Compressed Sensing , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[71]  Alexandre d'Aspremont,et al.  Testing the nullspace property using semidefinite programming , 2008, Math. Program..

[72]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[73]  Pablo A. Parrilo,et al.  Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization , 2007, SIAM Rev..

[74]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[75]  Vahid Tarokh,et al.  A Frame Construction and a Universal Distortion Bound for Sparse Representations , 2008, IEEE Transactions on Signal Processing.

[76]  Vivek K. Goyal,et al.  Simultaneously Sparse Solutions to Linear Inverse Problems with Multiple System Matrices and a Single Observation Vector , 2009, SIAM J. Sci. Comput..

[77]  B. Hassibi,et al.  Compressed sensing over the Grassmann manifold: A unified analytical framework , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[78]  Mihailo Stojnic,et al.  A simple performance analysis of ℓ1 optimization in compressed sensing , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[79]  Guillermo Sapiro,et al.  Discriminative learned dictionaries for local image analysis , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[80]  E. Candès.  The restricted isometry property and its implications for compressed sensing , 2008 .

[81]  R. Adamczak,et al.  Restricted Isometry Property of Matrices with Independent Columns and Neighborly Polytopes by Random Sampling , 2009, 0904.4723.

[82]  Yonina C. Eldar,et al.  Average Case Analysis of Multichannel Sparse Recovery Using Convex Relaxation , 2009, IEEE Transactions on Information Theory.

[83]  Joel A. Tropp,et al.  Algorithmic linear dimension reduction in the l_1 norm for sparse vectors , 2006, ArXiv.

[84]  Weiyu Xu,et al.  Necessary and sufficient conditions for success of the nuclear norm heuristic for rank minimization , 2008, 2008 47th IEEE Conference on Decision and Control.

[85]  A. Barvinok.  Approximating Orthogonal Matrices by Permutation Matrices , 2005, math/0510612.

[86]  Richard Baraniuk,et al.  Recovery of Clustered Sparse Signals from Compressive Measurements , 2009 .

[87]  M. Stojnic.  Various thresholds for $\ell_1$-optimization in compressed sensing , 2009 .

[88]  Bhaskar D. Rao,et al.  Sparse channel estimation via matching pursuit with application to equalization , 2002, IEEE Trans. Commun..

[89]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[90]  Vivek K. Goyal,et al.  Sparsity-Enforced Slice-Selective MRI RF Excitation Pulse Design , 2008, IEEE Transactions on Medical Imaging.

[91]  Arkadi Nemirovski,et al.  On verifiable sufficient conditions for sparse signal recovery via ℓ1 minimization , 2008, Math. Program..

[92]  Alexandros G. Dimakis,et al.  Sparse Recovery of Positive Signals with Minimal Expansion , 2009, ArXiv.

[93]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[94]  S. Foucart,et al.  Sparsest solutions of underdetermined linear systems via ℓq-minimization for 0 , 2009 .

[95]  V. Temlyakov.  A remark on simultaneous greedy approximation , 2004 .