Compressed Sensing Reconstruction via Belief Propagation

Compressed sensing is an emerging field that enables to reconstruct sparse or compressible signals from a small number of linear projections. We describe a specific measurement scheme using an LDPC-like measurement matrix, which is a real-valued analogue to LDPC techniques over a finite alphabet. We then describe the reconstruction details for mixture Gaussian signals. The technique can be extended to additional compressible signal models.

[1]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[2]  H. Sorenson,et al.  Recursive bayesian estimation using gaussian sums , 1971 .

[3]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[4]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[5]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[6]  A. Glavieux,et al.  Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1 , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[7]  David Leporini,et al.  Bayesian approach to best basis selection , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[8]  Brendan J. Frey,et al.  A Revolution: Belief Propagation in Graphs with Cycles , 1997, NIPS.

[9]  H. Chipman,et al.  Adaptive Bayesian Wavelet Shrinkage , 1997 .

[10]  D.J.C. MacKay,et al.  Good error-correcting codes based on very sparse matrices , 1997, Proceedings of IEEE International Symposium on Information Theory.

[11]  Brendan J. Frey,et al.  Graphical Models for Machine Learning and Digital Communication , 1998 .

[12]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[13]  B. Silverman,et al.  Wavelet thresholding via a Bayesian approach , 1998 .

[14]  Robert D. Nowak,et al.  Wavelet-based statistical signal processing using hidden Markov models , 1998, IEEE Trans. Signal Process..

[15]  Michael I. Jordan,et al.  Probabilistic Networks and Expert Systems , 1999 .

[16]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[17]  David W. Scott,et al.  From Kernels to Mixtures , 2001, Technometrics.

[18]  X. Jin.  Factor graphs and the Sum-Product Algorithm , 2002 .

[19]  Alan L. Yuille,et al.  CCCP Algorithms to Minimize the Bethe and Kikuchi Free Energies: Convergent Alternatives to Belief Propagation , 2002, Neural Computation.

[20]  Michael Mitzenmacher,et al.  A digital fountain approach to asynchronous reliable multicast , 2002, IEEE J. Sel. Areas Commun..

[21]  William T. Freeman,et al.  Nonparametric belief propagation , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[22]  William T. Freeman,et al.  Understanding belief propagation and its generalizations , 2003 .

[23]  S. Verdú,et al.  Noiseless Data Compression with Low-Density Parity-Check Codes , 2003, Advances in Network Information Theory.

[24]  M. Pretti,et al.  Stable propagation algorithm for the minimization of the Bethe free energy , 2003 .

[25]  Hilbert J. Kappen,et al.  Approximate Inference and Constrained Optimization , 2002, UAI.

[26]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[27]  Graham Cormode,et al.  Towards an Algorithmic Theory of Compressed Sensing , 2005 .

[28]  John W. Fisher,et al.  Loopy Belief Propagation: Convergence and Effects of Message Errors , 2005, J. Mach. Learn. Res..

[29]  Richard G. Baraniuk,et al.  Fast reconstruction of piecewise smooth signals from random projections , 2005 .

[30]  J. Tropp,et al.  SIGNAL RECOVERY FROM PARTIAL INFORMATION VIA ORTHOGONAL MATCHING PURSUIT , 2005 .

[31]  E. Candès,et al.  Error correction via linear programming , 2005, FOCS 2005.

[32]  D. Donoho,et al.  Neighborliness of randomly projected simplices in high dimensions. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[33]  M. Pretti.  A message-passing algorithm with damping , 2005 .

[34]  Richard G. Baraniuk,et al.  Sudocodes ߝ Fast Measurement and Reconstruction of Sparse Signals , 2006, 2006 IEEE International Symposium on Information Theory.

[35]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[36]  Graham Cormode,et al.  Combinatorial Algorithms for Compressed Sensing , 2006 .

[37]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[38]  Richard G. Baraniuk,et al.  Sparse Signal Detection from Incoherent Projections , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[39]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[40]  David L. Donoho,et al.  Sparse Solution Of Underdetermined Linear Equations By Stagewise Orthogonal Matching Pursuit , 2006 .

[41]  David L. Donoho,et al.  High-Dimensional Centrally Symmetric Polytopes with Neighborliness Proportional to Dimension , 2006, Discret. Comput. Geom..

[42]  Joel A. Tropp,et al.  Algorithmic linear dimension reduction in the l_1 norm for sparse vectors , 2006, ArXiv.

[43]  Lakhmi C. Jain,et al.  Introduction to Bayesian Networks , 2008 .