Upper-bounding ℓ1-optimization weak thresholds

In our recent work [49] we considered solving under-determi ned systems of linear equations with sparse solutions. In a large dimensional and statistical context w e proved that if the number of equations in the system is proportional to the length of the unknown vector th en there is a sparsity (number of non-zero elements of the unknown vector) also proportional to the length of t e unknown vector such that a polynomial l1-optimization technique succeeds in solving the system. We provided lower bounds on the proportionality constants that are in a solid numerical agreement with what o ne can observe through numerical experiments. Here we create a mechanism that can be used to derive the upper bounds on the proportionality constants. Moreover, the upper bounds obtained through such a mechanis m match the lower bounds from [49] and ultimately make the latter ones optimal.

[1]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[2]  David L. Donoho,et al.  Neighborly Polytopes And Sparse Solution Of Underdetermined Linear Equations , 2005 .

[3]  Y. Gordon On Milman's inequality and random subspaces which escape through a mesh in ℝ n , 1988 .

[4]  Babak Hassibi,et al.  Explicit measurements with almost optimal thresholds for compressed sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[5]  David L. Donoho,et al.  High-Dimensional Centrally Symmetric Polytopes with Neighborliness Proportional to Dimension , 2006, Discret. Comput. Geom..

[6]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[7]  Graham Cormode,et al.  Combinatorial Algorithms for Compressed Sensing , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[8]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[9]  D. Donoho,et al.  Neighborliness of randomly projected simplices in high dimensions. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Rayan Saab,et al.  Stable sparse approximations via nonconvex optimization , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[11]  B. Hassibi,et al.  Compressed sensing over the Grassmann manifold: A unified analytical framework , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[12]  Martin Vetterli,et al.  Sampling and reconstruction of signals with finite rate of innovation in the presence of noise , 2005, IEEE Transactions on Signal Processing.

[13]  Arkadi Nemirovski,et al.  On verifiable sufficient conditions for sparse signal recovery via ℓ1 minimization , 2008, Math. Program..

[14]  P Mcmullen,et al.  Non-linear angle-sum relations for polyhedral cones and polytopes , 1975, Mathematical Proceedings of the Cambridge Philosophical Society.

[15]  Venkatesh Saligrama,et al.  Thresholded Basis Pursuit: Quantizing Linear Programming Solutions for Optimal Support Recovery and Approximation in Compressed Sensing , 2008, ArXiv.

[16]  D. Donoho,et al.  Counting faces of randomly-projected polytopes when the projection radically lowers dimension , 2006, math/0607364.

[17]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[18]  Xiaoming Huo,et al.  Uncertainty principles and ideal atomic decomposition , 2001, IEEE Trans. Inf. Theory.

[19]  R. Adamczak,et al.  Restricted Isometry Property of Matrices with Independent Columns and Neighborly Polytopes by Random Sampling , 2009, 0904.4723.

[20]  Joel A. Tropp,et al.  Algorithmic linear dimension reduction in the l_1 norm for sparse vectors , 2006, ArXiv.

[21]  David L. Donoho,et al.  Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[22]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[23]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[24]  Alexandre d'Aspremont,et al.  Testing the nullspace property using semidefinite programming , 2008, Math. Program..

[25]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[26]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[27]  Vahid Tarokh,et al.  A Frame Construction and a Universal Distortion Bound for Sparse Representations , 2008, IEEE Transactions on Signal Processing.

[28]  Robert D. Nowak,et al.  Signal Reconstruction From Noisy Random Projections , 2006, IEEE Transactions on Information Theory.

[29]  Mihailo Stojnic,et al.  A rigorous geometry-probability equivalence in characterization of ℓ1-optimization , 2013, ArXiv.

[30]  R. Gribonval,et al.  Highly sparse representations from dictionaries are unique and independent of the sparseness measure , 2007 .

[31]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[32]  Yin Zhang Caam When is missing data recoverable ? , 2006 .

[33]  Piotr Indyk,et al.  Sparse Recovery Using Sparse Random Matrices , 2010, LATIN.

[34]  Mihailo Stojnic,et al.  A simple performance analysis of ℓ1 optimization in compressed sensing , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[35]  Graham Cormode,et al.  Combinatorial Algorithms for Compressed Sensing , 2006 .

[36]  Nathan Linial,et al.  How Neighborly Can a Centrally Symmetric Polytope Be? , 2006, Discret. Comput. Geom..

[37]  Arkadi Nemirovski,et al.  On sparse representation in pairs of bases , 2003, IEEE Trans. Inf. Theory.

[38]  Weiyu Xu,et al.  Efficient Compressive Sensing with Deterministic Guarantees Using Expander Graphs , 2007, 2007 IEEE Information Theory Workshop.

[39]  D. Donoho,et al.  Thresholds for the Recovery of Sparse Solutions via L1 Minimization , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[40]  Rémi Gribonval,et al.  Restricted Isometry Constants Where $\ell ^{p}$ Sparse Recovery Can Fail for $0≪ p \leq 1$ , 2009, IEEE Transactions on Information Theory.

[41]  Martin J. Wainwright,et al.  Sharp thresholds for high-dimensional and noisy recovery of sparsity , 2006, ArXiv.

[42]  D. Donoho,et al.  Sparse nonnegative solution of underdetermined linear equations by linear programming. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[43]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[44]  Mihailo Stojnic,et al.  Strong thresholds for ℓ2/ℓ1-optimization in block-sparse compressed sensing , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[45]  H. Ruben On the geometrical moments of skew-regular simplices in hyperspherical space, with some applications in geometry and mathematical statistics , 1960 .

[46]  J. K. Böröczky,et al.  Random projections of regular polytopes , 1999 .

[47]  Stéphane Chrétien,et al.  An Alternating $l_1$ Approach to the Compressed Sensing Problem , 2008, IEEE Signal Processing Letters.

[48]  Rolf Schneider,et al.  Random projections of regular simplices , 1992, Discret. Comput. Geom..

[49]  M. Rudelson,et al.  Geometric approach to error-correcting codes and reconstruction of signals , 2005, math/0502299.

[50]  Rémi Gribonval,et al.  Sparse representations in unions of bases , 2003, IEEE Trans. Inf. Theory.

[51]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[52]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[53]  David L. Donoho,et al.  Counting the Faces of Randomly-Projected Hypercubes and Orthants, with Applications , 2008, Discret. Comput. Geom..

[54]  R. Vershynin,et al.  One sketch for all: fast algorithms for compressed sensing , 2007, STOC '07.