Density Evolution Analysis of Node-Based Verification-Based Algorithms in Compressed Sensing

In this paper, we present a new approach for the analysis of iterative node-based verification-based (NB-VB) recovery algorithms in the context of compressed sensing. These algorithms are particularly interesting due to their low complexity (linear in the signal dimension n). The asymptotic analysis predicts the fraction of unverified signal elements at each iteration l in the asymptotic regime where n→∞. The analysis is similar in nature to the well-known density evolution technique commonly used to analyze iterative decoding algorithms. To perform the analysis, a message-passing interpretation of NB-VB algorithms is provided. This interpretation lacks the extrinsic nature of standard message-passing algorithms to which density evolution is usually applied. This requires a number of nontrivial modifications in the analysis. The analysis tracks the average performance of the recovery algorithms over the ensembles of input signals and sensing matrices as a function of l. Concentration results are devised to demonstrate that the performance of the recovery algorithms applied to any choice of the input signal over any realization of the sensing matrix follows the deterministic results of the analysis closely. Simulation results are also provided which demonstrate that the proposed asymptotic analysis matches the performance of recovery algorithms for large but finite values of n . Compared to the existing technique for the analysis of NB-VB algorithms, which is based on numerically solving a large system of coupled differential equations, the proposed method is more accurate and simpler to implement.

[1]  S. Mendelson,et al.  Uniform Uncertainty Principle for Bernoulli and Subgaussian Ensembles , 2006, math/0608665.

[2]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[3]  Emmanuel J. Candès,et al.  Decoding by linear programming , 2005, IEEE Transactions on Information Theory.

[4]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[5]  Deanna Needell,et al.  Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit , 2007, Found. Comput. Math..

[6]  Piotr Indyk.  Explicit constructions for compressed sensing of sparse signals , 2008, SODA '08.

[7]  Andrea Montanari,et al.  Counter braids: a novel counter architecture for per-flow measurement , 2008, SIGMETRICS '08.

[8]  Michael A. Saunders,et al.  Algorithm 583: LSQR: Sparse Linear Equations and Least Squares Problems , 1982, TOMS.

[9]  Joel A. Tropp,et al.  Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit , 2007, IEEE Transactions on Information Theory.

[10]  Noga Alon,et al.  The Probabilistic Method , 2015, Fundamentals of Ramsey Theory.

[11]  Andrea Montanari,et al.  Analysis of Belief Propagation for Non-Linear Problems: The Example of CDMA (or: How to Prove Tanaka's Formula) , 2006, 2006 IEEE Information Theory Workshop - ITW '06 Punta del Este.

[12]  Richard G. Baraniuk,et al.  Sudocodes ߝ Fast Measurement and Reconstruction of Sparse Signals , 2006, 2006 IEEE International Symposium on Information Theory.

[13]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[14]  Richard G. Baraniuk,et al.  Bayesian Compressive Sensing Via Belief Propagation , 2008, IEEE Transactions on Signal Processing.

[15]  Weiyu Xu,et al.  Efficient Compressive Sensing with Deterministic Guarantees Using Expander Graphs , 2007, 2007 IEEE Information Theory Workshop.

[16]  D. Donoho,et al.  Thresholds for the Recovery of Sparse Solutions via L1 Minimization , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[17]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[18]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[19]  Devavrat Shah,et al.  A simple message-passing algorithm for compressed sensing , 2010, 2010 IEEE International Symposium on Information Theory.

[20]  Joel A. Tropp,et al.  Algorithmic linear dimension reduction in the l_1 norm for sparse vectors , 2006, ArXiv.

[21]  Graham Cormode,et al.  Combinatorial Algorithms for Compressed Sensing , 2006 .

[22]  Fan Zhang,et al.  Compressed sensing and linear codes over real numbers , 2008, 2008 Information Theory and Applications Workshop.

[23]  Yi Lu,et al.  Counter Braids: Asymptotic optimality of the message passing decoding algorithm , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[24]  C. Robert.  The Bayesian choice : a decision-theoretic motivation , 1996 .

[25]  Michael Mitzenmacher,et al.  Verification-based decoding for packet-based low-density parity-check codes , 2005, IEEE Transactions on Information Theory.

[26]  Fan Zhang,et al.  Analysis of Verification-Based Decoding on the q -ary Symmetric Channel for Large q , 2008, IEEE Trans. Inf. Theory.

[27]  Fan Zhang,et al.  List-Message Passing Achieves Capacity on the q-ary Symmetric Channel for Large q , 2007, IEEE GLOBECOM 2007 - IEEE Global Telecommunications Conference.

[28]  Daniel A. Spielman,et al.  Improved low-density parity-check codes using irregular graphs and belief propagation , 1998, Proceedings. 1998 IEEE International Symposium on Information Theory (Cat. No.98CH36252).

[29]  Alain Glavieux,et al.  Reflections on the Prize Paper : "Near optimum error-correcting coding and decoding: turbo codes" , 1998 .

[30]  Harry L. Van Trees,et al.  Detection, Estimation, and Modulation Theory, Part I , 1968 .

[31]  Daniel A. Spielman,et al.  Efficient erasure correcting codes , 2001, IEEE Trans. Inf. Theory.

[32]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[33]  R. DeVore,et al.  A Simple Proof of the Restricted Isometry Property for Random Matrices , 2008 .

[34]  R. Vershynin,et al.  One sketch for all: fast algorithms for compressed sensing , 2007, STOC '07.

[35]  Chih-Chun Wang,et al.  Multiuser Detection of Sparsely Spread CDMA , 2008, IEEE Journal on Selected Areas in Communications.

[36]  Fan Zhang,et al.  On the Iterative Decoding of High-Rate LDPC Codes With Applications in Compressed Sensing , 2009, ArXiv.

[37]  Sergio Verdú,et al.  Fundamental limits of almost lossless analog compression , 2009, 2009 IEEE International Symposium on Information Theory.

[38]  Dongning Guo,et al.  A single-letter characterization of optimal noisy compressed sensing , 2009, 2009 47th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[39]  Piotr Indyk,et al.  Combining geometry and combinatorics: A unified approach to sparse signal recovery , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[40]  V. Chandar.  A Negative Result Concerning Explicit Matrices With The Restricted Isometry Property , 2008 .

[41]  Andrea Montanari,et al.  The dynamics of message passing on dense graphs, with applications to compressed sensing , 2010, 2010 IEEE International Symposium on Information Theory.

[42]  Joel A. Tropp,et al.  Topics in sparse approximation , 2004 .

[43]  Vahid Tarokh,et al.  Low density frames for compressive sensing , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.