Model-based Sketching and Recovery with Expanders

Linear sketching and recovery of sparse vectors with randomly constructed sparse matrices has numerous applications in several areas, including compressive sensing, data stream computing, graph sketching, and combinatorial group testing. This paper considers the same problem with the added twist that the sparse coefficients of the unknown vector exhibit further correlations as determined by a known sparsity model. We prove that exploiting model-based sparsity in recovery provably reduces the sketch size without sacrificing recovery quality. In this context, we present the model-expander iterative hard thresholding algorithm for recovering model sparse signals from linear sketches obtained via sparse adjacency matrices of expander graphs with rigorous performance guarantees. The main computational cost of our algorithm depends on the difficulty of projecting onto the model-sparse set. For the tree and group-based sparsity models we describe in this paper, such projections can be obtained in linear time. Finally, we provide numerical experiments to illustrate the theoretical results in action.

[1]  Massimo Fornasier,et al.  Recovery Algorithms for Vector-Valued Data with Joint Sparsity Constraints , 2008, SIAM J. Numer. Anal..

[2]  Sudipto Guha,et al.  Graph sketches: sparsification, spanners, and subgraphs , 2012, PODS.

[3]  Douglas L. Jones,et al.  A signal-dependent time-frequency representation: fast algorithm for optimal kernel design , 1994, IEEE Trans. Signal Process..

[4]  Avi Wigderson,et al.  Randomness conductors and constant-degree lossless expanders , 2002, STOC '02.

[5]  Holger Rauhut,et al.  A Mathematical Introduction to Compressive Sensing , 2013, Applied and Numerical Harmonic Analysis.

[6]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[7]  Piotr Indyk,et al.  On Model-Based RIP-1 Matrices , 2013, ICALP.

[8]  D K Smith,et al.  Numerical Optimization , 2001, J. Oper. Res. Soc..

[9]  慧 廣瀬.  A Mathematical Introduction to Compressive Sensing , 2015 .

[10]  Volkan Cevher,et al.  Model-Based Compressive Sensing , 2008, IEEE Transactions on Information Theory.

[11]  Eric Price,et al.  Efficient sketches for the set query problem , 2010, SODA '11.

[12]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[13]  Enkatesan G Uruswami.  Unbalanced expanders and randomness extractors from Parvaresh-Vardy codes , 2008 .

[14]  Yonina C. Eldar,et al.  Robust Recovery of Signals From a Structured Union of Subspaces , 2008, IEEE Transactions on Information Theory.

[15]  David P. Woodruff,et al.  Open Problems in Data Streams, Property Testing, and Related Topics , 2011 .

[16]  Jared Tanner,et al.  Vanishingly Sparse Matrices and Expander Graphs, With Application to Compressed Sensing , 2012, IEEE Transactions on Information Theory.

[17]  Julien Mairal,et al.  Proximal Methods for Hierarchical Sparse Coding , 2010, J. Mach. Learn. Res..

[18]  Matthieu Kowalski,et al.  Improving M/EEG source localizationwith an inter-condition sparse prior , 2009, 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro.

[19]  Peter Bro Miltersen,et al.  Are bitvectors optimal? , 2000, STOC '00.

[20]  Wolfdieter Lang,et al.  Combinatorial Interpretation of Generalized Stirling Numbers , 2009 .

[21]  S. Muthukrishnan,et al.  Data streams: algorithms and applications , 2005, SODA '03.

[22]  Anna C. Gilbert,et al.  Compressing Network Graphs , 2004 .

[23]  Nikhil S. Rao,et al.  Signal Recovery in Unions of Subspaces with Applications to Compressive Imaging , 2012, 1209.3079.

[24]  Jean-Philippe Vert,et al.  Group lasso with overlap and graph lasso , 2009, ICML '09.

[25]  Richard G. Baraniuk,et al.  Joint Sparsity Models for Distributed Compressed Sensing , 2005 .

[26]  Babak Hassibi,et al.  On the Reconstruction of Block-Sparse Signals With an Optimal Number of Measurements , 2008, IEEE Transactions on Signal Processing.

[27]  P. Indyk,et al.  Near-Optimal Sparse Recovery in the L1 Norm , 2008, 2008 49th Annual IEEE Symposium on Foundations of Computer Science.

[29]  Marco F. Duarte,et al.  Compressive sensing recovery of spike trains using a structured sparsity model , 2009 .

[30]  Cynthia Dwork,et al.  The price of privacy and the limits of LP decoding , 2007, STOC '07.

[31]  S. Frick,et al.  Compressed Sensing , 2014, Computer Vision, A Reference Guide.

[32]  Piotr Indyk,et al.  K-median clustering, model-based compressive sensing, and sparse recovery for earth mover distance , 2011, STOC '11.

[33]  Piotr Indyk,et al.  Sparse Recovery Using Sparse Matrices , 2010, Proceedings of the IEEE.

[34]  Coralia Cartis,et al.  An Exact Tree Projection Algorithm for Wavelets , 2013, IEEE Signal Processing Letters.

[35]  Piotr Indyk,et al.  Combining geometry and combinatorics: A unified approach to sparse signal recovery , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[36]  Uriel Feige,et al.  Proceedings of the thirty-ninth annual ACM symposium on Theory of computing , 2007, STOC 2007.

[37]  Volkan Cevher,et al.  Group-Sparse Model Selection: Hardness and Relaxations , 2013, IEEE Transactions on Information Theory.

[38]  A. Robert Calderbank,et al.  Efficient and Robust Compressed Sensing Using Optimized Expander Graphs , 2009, IEEE Transactions on Information Theory.