Detection-Directed Sparse Estimation using Bayesian Hypothesis Test and Belief Propagation

In this paper, we propose a sparse recovery algorithm called detection-directed (DD) sparse estimation using Bayesian hypothesis test (BHT) and belief propagation (BP). In this framework, we consider the use of sparse-binary sensing matrices which has the tree-like property and the sampled-message approach for the implementation of BP. The key idea behind the proposed algorithm is that the recovery takes DD-estimation structure consisting of two parts: support detection and signal value estimation. BP and BHT perform the support detection, and an MMSE estimator finds the signal values using the detected support set. The proposed algorithm provides noise-robustness against measurement noise beyond the conventional MAP approach, as well as a solution to remove quantization effect by the sampled-message based BP independently of memory size for the message sampling. We explain how the proposed algorithm can have the aforementioned characteristics via exemplary discussion. In addition, our experiments validate such superiority of the proposed algorithm, compared to recent algorithms under noisy setup. Interestingly the experimental results show that performance of the proposed algorithm approaches that of the oracle estimator as SNR becomes higher.

[1]  Aggelos K. Katsaggelos,et al.  Bayesian Compressive Sensing Using Laplace Priors , 2010, IEEE Transactions on Image Processing.

[2]  Chih-Chun Wang,et al.  Multiuser Detection of Sparsely Spread CDMA , 2008, IEEE Journal on Selected Areas in Communications.

[3]  Aleksandra Pizurica,et al.  Estimating the probability of the presence of a signal of interest in multiresolution single- and multiband image denoising , 2006, IEEE Transactions on Image Processing.

[4]  Joel A. Tropp,et al.  Just relax: convex programming methods for identifying sparse signals in noise , 2006, IEEE Transactions on Information Theory.

[5]  Michael Elad,et al.  Closed-Form MMSE Estimation for Signal Denoising Under Sparse Representation Modeling Over a Unitary Dictionary , 2010, IEEE Transactions on Signal Processing.

[6]  B. Vidakovic Nonlinear wavelet shrinkage with Bayes rules and Bayes factors , 1998 .

[7]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[8]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[9]  Chih-Chun Wang,et al.  Random Sparse Linear Systems Observed Via Arbitrary Channels: A Decoupling Principle , 2007, 2007 IEEE International Symposium on Information Theory.

[10]  Sundeep Rangan,et al.  Generalized approximate message passing for estimation with random linear mixing , 2010, 2011 IEEE International Symposium on Information Theory Proceedings.

[11]  S. Kay Fundamentals of statistical signal processing: estimation theory , 1993 .

[12]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevan e Ve tor Ma hine , 2001 .

[13]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparse Signal Recovery: Dense versus Sparse Measurement Matrices , 2008, IEEE Transactions on Information Theory.

[14]  J. S. Rao,et al.  Spike and slab variable selection: Frequentist and Bayesian strategies , 2005, math/0505633.

[15]  Giancarlo Prati,et al.  Blind Equalization and Carrier Recovery Using a "Stop-and-Go" Decision-Directed Algorithm , 1987, IEEE Trans. Commun..

[16]  Vahid Tarokh,et al.  A Coding Theory Approach to Noisy Compressive Sensing Using Low Density Frames , 2011, IEEE Transactions on Signal Processing.

[17]  Richard Baraniuk,et al.  Compressed Sensing Reconstruction via Belief Propagation , 2006 .

[18]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[19]  Kiseon Kim,et al.  Bayesian hypothesis test for sparse support recovery using belief propagation , 2012, 2012 IEEE Statistical Signal Processing Workshop (SSP).

[20]  Robert D. Nowak,et al.  Signal Reconstruction From Noisy Random Projections , 2006, IEEE Transactions on Information Theory.

[21]  Dongning Guo,et al.  Asymptotic Mean-Square Optimality of Belief Propagation for Sparse Linear Systems , 2006, 2006 IEEE Information Theory Workshop - ITW '06 Chengdu.

[22]  Sundeep Rangan,et al.  Estimation with random linear mixing, belief propagation and compressed sensing , 2010, 2010 44th Annual Conference on Information Sciences and Systems (CISS).

[23]  Kiseon Kim,et al.  Message Passing Aided Least Square Recovery for Compressive Sensing , 2011 .

[24]  Andrea Montanari,et al.  The dynamics of message passing on dense graphs, with applications to compressed sensing , 2010, 2010 IEEE International Symposium on Information Theory.

[25]  Richard G. Baraniuk,et al.  Bayesian Compressive Sensing Via Belief Propagation , 2008, IEEE Transactions on Signal Processing.

[26]  Martin J. Wainwright,et al.  Quantized stochastic belief propagation: Efficient message-passing for continuous state spaces , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[27]  Martin J. Wainwright,et al.  Information-Theoretic Limits on Sparsity Recovery in the High-Dimensional and Noisy Setting , 2007, IEEE Transactions on Information Theory.

[28]  Sergio Verdú,et al.  Randomly spread CDMA: asymptotics via statistical physics , 2005, IEEE Transactions on Information Theory.

[29]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[30]  E. Candès,et al.  Stable signal recovery from incomplete and inaccurate measurements , 2005, math/0503066.

[31]  Andrea Montanari,et al.  Message passing algorithms for compressed sensing: I. motivation and construction , 2009, 2010 IEEE Information Theory Workshop on Information Theory (ITW 2010, Cairo).

[32]  D. Godard,et al.  Self-Recovering Equalization and Carrier Tracking in Two-Dimensional Data Communication Systems , 1980, IEEE Trans. Commun..

[33]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.

[34]  Michael Elad,et al.  Stable recovery of sparse overcomplete representations in the presence of noise , 2006, IEEE Transactions on Information Theory.

[35]  Michael Elad,et al.  A Plurality of Sparse Representations Is Better Than the Sparsest One Alone , 2009, IEEE Transactions on Information Theory.

[36]  Andrea Montanari,et al.  Message-passing algorithms for compressed sensing , 2009, Proceedings of the National Academy of Sciences.

[37]  J. Navarro-Pedreño Numerical Methods for Least Squares Problems , 1996 .

[38]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[39]  Ephraim Speech enhancement using a minimum mean square error short-time spectral amplitude estimator , 1984 .

[40]  Terence Tao,et al.  The Dantzig selector: Statistical estimation when P is much larger than n , 2005, math/0506081.

[41]  F. Pukelsheim The Three Sigma Rule , 1994 .

[42]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[43]  Justin Ziniel,et al.  Fast bayesian matching pursuit , 2008, 2008 Information Theory and Applications Workshop.

[44]  Andrea Montanari,et al.  The Noise-Sensitivity Phase Transition in Compressed Sensing , 2010, IEEE Transactions on Information Theory.

[45]  David Middleton,et al.  Simultaneous optimum detection and estimation of signals in noise , 1968, IEEE Trans. Inf. Theory.

[46]  William T. Freeman,et al.  Nonparametric belief propagation , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[47]  David J. C. MacKay,et al.  Good Error-Correcting Codes Based on Very Sparse Matrices , 1997, IEEE Trans. Inf. Theory.

[48]  Jian Li,et al.  Computationally Efficient Sparse Bayesian Learning via Belief Propagation , 2009, IEEE Transactions on Signal Processing.