Structured sublinear compressive sensing via belief propagation

Abstract Compressive sensing (CS) is a sampling technique designed for reducing the complexity of sparse data acquisition. One of the major obstacles for practical deployment of CS techniques is the signal reconstruction time and the high storage cost of random sensing matrices. We propose a new structured compressive sensing scheme, based on codes of graphs, that allows for a joint design of structured sensing matrices and logarithmic-complexity reconstruction algorithms. The compressive sensing matrices can be shown to offer asymptotically optimal performance when used in combination with orthogonal matching pursuit (OMP) methods. For reduced-complexity greedy reconstruction schemes, we propose a new family of list-decoding belief propagation algorithms, as well as reinforced and multiple-basis belief propagation (BP) algorithms. Our simulation results indicate that reinforced BP CS schemes offer very good complexity–performance tradeoffs for very sparse signal vectors.

[1]  Emmanuel J. Candès,et al.  Error correction via linear programming , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[2]  Vahid Tarokh,et al.  Spectral Distribution of Random Matrices From Binary Linear Block Codes , 2011, IEEE Transactions on Information Theory.

[3]  Y. Weiss,et al.  Finding the M Most Probable Configurations using Loopy Belief Propagation , 2003, NIPS 2003.

[4]  R. Varga Geršgorin And His Circles , 2004 .

[5]  Richard G. Baraniuk,et al.  Bayesian Compressive Sensing Via Belief Propagation , 2008, IEEE Transactions on Signal Processing.

[6]  Piotr Indyk,et al.  Sparse Recovery Using Sparse Matrices , 2010, Proceedings of the IEEE.

[7]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[8]  Deanna Needell,et al.  Signal Recovery From Incomplete and Inaccurate Measurements Via Regularized Orthogonal Matching Pursuit , 2007, IEEE Journal of Selected Topics in Signal Processing.

[9]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[10]  Vahid Tarokh,et al.  Random frames from binary linear block codes , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[11]  Johannes B. Huber Iterated Soft-Decision Interfer-ence Cancellation for CDMA , 1998 .

[12]  Graham Cormode,et al.  Combinatorial Algorithms for Compressed Sensing , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[13]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[14]  Piotr Indyk,et al.  Combining geometry and combinatorics: A unified approach to sparse signal recovery , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[15]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[16]  A. Robert Calderbank,et al.  Sparse reconstruction via the Reed-Muller Sieve , 2010, 2010 IEEE International Symposium on Information Theory.

[17]  Vladimir N. Temlyakov,et al.  On the size of incoherent systems , 2011, J. Approx. Theory.

[18]  Emmanuel J. Candès,et al.  Near-Optimal Signal Recovery From Random Projections: Universal Encoding Strategies? , 2004, IEEE Transactions on Information Theory.

[19]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[20]  Robert D. Nowak,et al.  Toeplitz Compressed Sensing Matrices With Applications to Sparse Channel Estimation , 2010, IEEE Transactions on Information Theory.

[21]  A. Robert Calderbank,et al.  Efficient and Robust Compressed Sensing Using Optimized Expander Graphs , 2009, IEEE Transactions on Information Theory.

[22]  Alexandros G. Dimakis,et al.  LP decoding meets LP decoding: A connection between channel coding and compressed sensing , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[23]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[24]  E. Candès The restricted isometry property and its implications for compressed sensing , 2008 .

[25]  Johannes B. Huber,et al.  Multiple-bases belief-propagation decoding of high-density cyclic codes , 2009, IEEE Transactions on Communications.

[26]  Joel A. Tropp,et al.  Greed is good: algorithmic results for sparse approximation , 2004, IEEE Transactions on Information Theory.

[27]  O. Milenkovic,et al.  The Third Support Weight Enumerators of the Doubly-Even, Self-Dual Codes , 2003 .

[28]  Deanna Needell,et al.  Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit , 2007, Found. Comput. Math..

[29]  Olgica Milenkovic,et al.  Subspace Pursuit for Compressive Sensing Signal Reconstruction , 2008, IEEE Transactions on Information Theory.

[30]  Olgica Milenkovic,et al.  The third support weight enumerators of the doubly-even, self-dual [32, 16, 8] codes , 2003, IEEE Trans. Inf. Theory.

[31]  A. Robert Calderbank,et al.  Construction of a Large Class of Deterministic Sensing Matrices That Satisfy a Statistical Isometry Property , 2009, IEEE Journal of Selected Topics in Signal Processing.

[32]  Anmin Huang,et al.  A fast subspace pursuit for compressive sensing , 2009 .

[33]  Yoram Bresler,et al.  Sub-Nyquist sampling of multiband signals: perfect reconstruction and bounds on aliasing error , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[34]  Riccardo Zecchina,et al.  Encoding for the Blackwell Channel with Reinforced Belief Propagation , 2007, 2007 IEEE International Symposium on Information Theory.

[35]  Olgica Milenkovic,et al.  Sublinear compressive sensing reconstruction via belief propagation decoding , 2009, 2009 IEEE International Symposium on Information Theory.

[36]  Ronald A. DeVore,et al.  Deterministic constructions of compressed sensing matrices , 2007, J. Complex..

[37]  Emmanuel J. Cand The Restricted Isometry Property and Its Implications for Compressed Sensing , 2008 .

[38]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[39]  I. G. Núñez,et al.  Generalized Hamming Weights for Linear Codes , 2001 .

[40]  Stephen J. Dilworth,et al.  Explicit constructions of RIP matrices and related problems , 2010, ArXiv.

[41]  Evangelos Eleftheriou,et al.  Regular and irregular progressive edge-growth tanner graphs , 2005, IEEE Transactions on Information Theory.

[42]  Richard G. Baraniuk,et al.  Sudocodes ߝ Fast Measurement and Reconstruction of Sparse Signals , 2006, 2006 IEEE International Symposium on Information Theory.

[43]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[44]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[45]  Radford M. Neal,et al.  Near Shannon limit performance of low density parity check codes , 1996 .

[46]  J. Tropp,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, Commun. ACM.

[47]  Simon Litsyn,et al.  On ensembles of low-density parity-check codes: Asymptotic distance distributions , 2002, IEEE Trans. Inf. Theory.

[48]  R. Vershynin,et al.  One sketch for all: fast algorithms for compressed sensing , 2007, STOC '07.