Bernoulli-Gaussian Approximate Message-Passing Algorithm for Compressed Sensing with 1D-Finite-Difference Sparsity

This paper proposes a fast approximate message-passing (AMP) algorithm for solving compressed sensing (CS) recovery problems with 1D-finite-difference sparsity in term of MMSE estimation. The proposed algorithm, named ssAMP-BGFD, is low-computational with its fast convergence and cheap per-iteration cost, providing phase transition nearly approaching to the state-of-the-art. The proposed algorithm is originated from a sum-product message-passing rule, applying a Bernoulli-Gaussian (BG) prior, seeking an MMSE solution. The algorithm construction includes not only the conventional AMP technique for the measurement fidelity, but also suggests a simplified message-passing method to promote the signal sparsity in finite-difference. Furthermore, we provide an EM-tuning methodology to learn the BG prior parameters, suggesting how to use some practical measurement matrices satisfying the RIP requirement under the ssAMP-BGFD recovery. Extensive empirical results confirms performance of the proposed algorithm, in phase transition, convergence speed, and CPU runtime, compared to the recent algorithms.

[1]  Alexandros G. Dimakis,et al.  LDPC Codes for Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[2]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[3]  Florent Krzakala,et al.  On convergence of approximate message passing , 2014, 2014 IEEE International Symposium on Information Theory.

[4]  Truong Q. Nguyen,et al.  An Augmented Lagrangian Method for Total Variation Video Restoration , 2011, IEEE Transactions on Image Processing.

[5]  Laurent Condat,et al.  A Direct Algorithm for 1-D Total Variation Denoising , 2013, IEEE Signal Processing Letters.

[6]  Andrea Montanari,et al.  Message passing algorithms for compressed sensing: I. motivation and construction , 2009, 2010 IEEE Information Theory Workshop on Information Theory (ITW 2010, Cairo).

[7]  Florent Krzakala,et al.  Statistical physics-based reconstruction in compressed sensing , 2011, ArXiv.

[8]  Richard G. Baraniuk,et al.  Asymptotic Analysis of Complex LASSO via Complex Approximate Message Passing (CAMP) , 2011, IEEE Transactions on Information Theory.

[9]  Michael Unser,et al.  The analog formulation of sparsity implies infinite divisibility and rules out Bernoulli-Gaussian priors , 2012, 2012 IEEE Information Theory Workshop.

[10]  Michio Homma,et al.  Direct observation of steps in rotation of the bacterial flagellar motor , 2005, Nature.

[11]  Max A. Little,et al.  Sparse Bayesian step-filtering for high-throughput analysis of molecular machine dynamics , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[12]  Jieping Ye,et al.  An efficient algorithm for a class of fused lasso problems , 2010, KDD.

[13]  P. Schellhammer,et al.  Serum protein fingerprinting coupled with a pattern-matching algorithm distinguishes prostate cancer from benign prostate hyperplasia and healthy men. , 2002, Cancer research.

[14]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[15]  Michael Elad,et al.  The Cosparse Analysis Model and Algorithms , 2011, ArXiv.

[16]  Richard G. Baraniuk,et al.  Bayesian Compressive Sensing Via Belief Propagation , 2008, IEEE Transactions on Signal Processing.

[17]  Emmanuel Barillot,et al.  Analysis of array CGH data: from signal ratio to gain and loss of DNA regions , 2004, Bioinform..

[18]  William T. Freeman,et al.  Nonparametric belief propagation , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[19]  Sundeep Rangan,et al.  On the convergence of approximate message passing with arbitrary matrices , 2014, 2014 IEEE International Symposium on Information Theory.

[20]  Marc Teboulle,et al.  A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems , 2009, SIAM J. Imaging Sci..

[21]  Philip Schniter,et al.  Compressive Imaging Using Approximate Message Passing and a Markov-Tree Prior , 2010, IEEE Transactions on Signal Processing.

[22]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[23]  Yonina C. Eldar,et al.  Compressed Sensing with Coherent and Redundant Dictionaries , 2010, ArXiv.

[24]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[25]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[26]  Philip Schniter,et al.  Generalized Approximate Message Passing for the Cosparse Analysis Model , 2013, ArXiv.

[27]  Yin Zhang,et al.  A Compressive Sensing and Unmixing Scheme for Hyperspectral Data Processing , 2012, IEEE Transactions on Image Processing.

[28]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[29]  J. S. Rao,et al.  Spike and slab variable selection: Frequentist and Bayesian strategies , 2005, math/0505633.

[30]  Yanting Ma,et al.  Compressive Imaging via Approximate Message Passing With Image Denoising , 2014, IEEE Transactions on Signal Processing.

[31]  Emmanuel J. Candès,et al.  Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information , 2004, IEEE Transactions on Information Theory.

[32]  E. Lander,et al.  Assessing the significance of chromosomal aberrations in cancer: Methodology and application to glioma , 2007, Proceedings of the National Academy of Sciences.

[33]  Andrea Montanari,et al.  Accurate Prediction of Phase Transitions in Compressed Sensing via a Connection to Minimax Denoising , 2011, IEEE Transactions on Information Theory.

[34]  Georgios B. Giannakis,et al.  Compressed Sensing for Wideband Cognitive Radios , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[35]  Rachel Ward,et al.  New and Improved Johnson-Lindenstrauss Embeddings via the Restricted Isometry Property , 2010, SIAM J. Math. Anal..

[36]  Max A. Little,et al.  Generalized methods and solvers for noise removal from piecewise constant signals. I. Background theory , 2011, Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[37]  Richard G. Baraniuk,et al.  VLSI Design of Approximate Message Passing for Signal Restoration and Compressive Sensing , 2012, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[38]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[39]  Sundeep Rangan,et al.  Generalized approximate message passing for cosparse analysis compressive sensing , 2013, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[40]  Ronald A. DeVore,et al.  Deterministic constructions of compressed sensing matrices , 2007, J. Complex..

[41]  Philip Schniter,et al.  Expectation-Maximization Gaussian-Mixture Approximate Message Passing , 2012, IEEE Transactions on Signal Processing.

[42]  John R. Gilbert,et al.  Sparse Matrices in MATLAB: Design and Implementation , 1992, SIAM J. Matrix Anal. Appl..

[43]  Ying Wang,et al.  Compressive wide-band spectrum sensing , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[44]  Sundeep Rangan,et al.  Hybrid generalized approximate message passing with applications to structured sparsity , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[45]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[46]  Y. Nesterov.  A method for solving the convex programming problem with convergence rate O(1/k^2) , 1983 .

[47]  Kiseon Kim,et al.  Bayesian Hypothesis Test Using Nonparametric Belief Propagation for Noisy Sparse Recovery , 2015, IEEE Transactions on Signal Processing.

[48]  Richard G. Baraniuk,et al.  Random Filters for Compressive Sampling and Reconstruction , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[49]  Vahid Tarokh,et al.  A Coding Theory Approach to Noisy Compressive Sensing Using Low Density Frames , 2011, IEEE Transactions on Signal Processing.

[50]  Michael Unser,et al.  Approximate Message Passing With Consistent Parameter Estimation and Applications to Sparse Learning , 2012, IEEE Transactions on Information Theory.

[51]  Andrea Montanari,et al.  Universality in Polytope Phase Transitions and Message Passing Algorithms , 2012, ArXiv.

[52]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[53]  Hyunju Lee,et al.  Wavelet-based identification of DNA focal genomic aberrations from single nucleotide polymorphism arrays , 2011, BMC Bioinformatics.

[54]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[55]  Richard G. Baraniuk,et al.  From Denoising to Compressed Sensing , 2014, IEEE Transactions on Information Theory.

[56]  Sundeep Rangan,et al.  Generalized approximate message passing for estimation with random linear mixing , 2010, 2011 IEEE International Symposium on Information Theory Proceedings.

[57]  Stephen J. Wright,et al.  Toeplitz-Structured Compressed Sensing Matrices , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[58]  Stephen Chak Tornay.  Ockham: studies and selections, , 1938 .

[59]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[60]  Sundeep Rangan,et al.  Adaptive damping and mean removal for the generalized approximate message passing algorithm , 2014, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[61]  Michael Elad,et al.  On MAP and MMSE estimators for the co-sparse analysis model , 2014, Digit. Signal Process..

[62]  Andrea Montanari,et al.  The dynamics of message passing on dense graphs, with applications to compressed sensing , 2010, 2010 IEEE International Symposium on Information Theory.

[63]  Sundeep Rangan,et al.  Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization , 2015, IEEE Transactions on Information Theory.

[64]  R. Tibshirani,et al.  PATHWISE COORDINATE OPTIMIZATION , 2007, 0708.1485.

[65]  Andrea Montanari,et al.  Graphical Models Concepts in Compressed Sensing , 2010, Compressed Sensing.

[66]  Farrokh Marvasti,et al.  Deterministic Construction of Binary, Bipolar, and Ternary Compressed Sensing Matrices , 2009, IEEE Transactions on Information Theory.

[67]  Daniela M Witten,et al.  Extensions of Sparse Canonical Correlation Analysis with Applications to Genomic Data , 2009, Statistical applications in genetics and molecular biology.

[68]  Richard G. Baraniuk,et al.  Parameterless Optimal Approximate Message Passing , 2013, ArXiv.

[69]  R. Tibshirani,et al.  A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. , 2009, Biostatistics.

[70]  Junfeng Yang,et al.  A New Alternating Minimization Algorithm for Total Variation Image Reconstruction , 2008, SIAM J. Imaging Sci..

[71]  Florent Krzakala,et al.  Sparse Estimation with the Swept Approximated Message-Passing Algorithm , 2014, ArXiv.