Lower bounds for sparse recovery

We consider the following <i>k</i>-sparse recovery problem: design an <i>m</i> x <i>n</i> matrix <i>A</i>, such that for any signal <i>x</i>, given <i>Ax</i> we can efficiently recover x satisfying ||<i>x</i> -- x||<sub>i</sub> ≤ <i>C</i> min<sub><i>k</i></sub>-sparse <i>x</i>' ||<i>x</i> - <i>x</i>'||<sub>1</sub>. It is known that there exist matrices A with this property that have only <i>O</i>(<i>k</i> log(<i>n/k</i>)) rows. In this paper we show that this bound is tight. Our bound holds even for the more general <i>randomized</i> version of the problem, where <i>A</i> is a random variable, and the recovery algorithm is required to work for any fixed x with constant probability (over <i>A</i>).

[1]  Ting Sun,et al.  Single-pixel imaging via compressive sampling , 2008, IEEE Signal Process. Mag..

[2]  E. Gluskin.  NORMS OF RANDOM MATRICES AND WIDTHS OF FINITE-DIMENSIONAL SETS , 1984 .

[3]  A. Razborov.  Communication Complexity , 2011 .

[4]  Peter Bro Miltersen,et al.  On data structures and asymmetric communication complexity , 1994, STOC '95.

[5]  David P. Woodruff,et al.  On the exact space complexity of sketching and streaming small norms , 2010, SODA '10.

[6]  Piotr Indyk,et al.  Combining geometry and combinatorics: A unified approach to sparse signal recovery , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[7]  B. S. Kašin,et al.  DIAMETERS OF SOME FINITE-DIMENSIONAL SETS AND CLASSES OF SMOOTH FUNCTIONS , 1977 .

[8]  Laszlo GYijRFI,et al.  Superimposed Codes in R " , 1986 .

[9]  Moses Charikar,et al.  Finding frequent items in data streams , 2002, Theor. Comput. Sci..

[10]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting , 2007, IEEE Transactions on Information Theory.

[11]  Sumit Ganguly,et al.  Finding Frequent Items over General Update Streams , 2008, SSDBM.

[12]  Yuan Zhou.  Introduction to Coding Theory , 2010 .

[13]  David P. Woodruff,et al.  Numerical linear algebra in the streaming model , 2009, STOC '09.

[14]  Noam Nisan,et al.  On Randomized One-round Communication Complexity , 1995, STOC '95.

[15]  WigdersonAvi,et al.  On Data Structures and Asymmetric Communication Complexity , 1998 .

[16]  Olgica Milenkovic,et al.  Weighted Superimposed Codes and Constrained Integer Compressed Sensing , 2008, IEEE Transactions on Information Theory.

[17]  Robert Krauthgamer,et al.  The Sketching Complexity of Pattern Matching , 2004, APPROX-RANDOM.

[18]  Graham Cormode,et al.  An improved data stream summary: the count-min sketch and its applications , 2004, J. Algorithms.

[19]  Holger Rauhut,et al.  The Gelfand widths of lp-balls for 0p<=1 , 2010, J. Complex..

[20]  J. H. van Lint,et al.  Introduction to Coding Theory , 1982 .

[21]  László Györfi,et al.  Superimposed codes in Rn , 1988, IEEE Trans. Inf. Theory.

[22]  Andrej Yu. Garnaev,et al.  On widths of the Euclidean Ball , 1984 .

[23]  Zoltán Füredi,et al.  An Improved Upper Bound of the Rate of Euclidean Superimposed Codes , 1999, IEEE Trans. Inf. Theory.

[24]  Piotr Indyk,et al.  Nearest-neighbor-preserving embeddings , 2007, TALG.

[25]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[26]  V. Temlyakov,et al.  A remark on Compressed Sensing , 2007 .