2 2 A ug 2 01 1 1 Sparse Estimation using Bayesian Hierarchical Prior Modeling for Real and Complex Models

Sparse modeling and estimation of complex signals is not unc ommon in practice. However, historically, much attention has been drawn to real-valued sys tem models, lacking the research of sparse signal modeling and estimation for complex-valued models. This paper introduces a unifying sparse Bayesian formalism that generalizes to complexas well as r e l-valued systems. The methodology relies on hierarchical Bayesian sparsity-inducing prior modelin g of the parameter of interest. This approach allows for the Bayesian modeling of l1-norm constraint for complex-valued as well as real-valued mo els. In addition, the proposed two-layer hierarchical model all ows for the design of novel priors for sparse estimation that outperform the Bayesian formulation of the l1-norm constraint and lead to estimators approximating a soft-thresholding rule. An extension of th e wo-layer model to a three-layer model is also presented. Varying the free parameters of the three-la yer model leads to estimators that approximate a hard-thresholding rule. Finally, a variational messagepassing (VMP) implementation of the proposed Bayesian method that effectively exploits the hierarchica l structure of the inference problem is presented. The simulation results show that the VMP algorithm outperfo rms existing sparse methods both in terms of the sparsity of the estimation results and achieved mean s quared error in low and moderate SNR regimes. This work was supported in part by the 4GMCT cooperative rese arch project funded by Intel Mobile Communications, Agilen t Technologies, Aalborg University and the Danish National A dvanced Technology Foundation. This research was also supp orted in part by the project ICT248894 Wireless Hybrid Enhanced M obile Radio Estimators (WHERE2) and by Erwin Schrödinger Postdoctoral Fellowship, Austrian Science Fund (FWF) Proj ect J2909-N23. This work has been submitted to the IEEE for possible publica tion. Copyright may be transferred without notice, after wh ich this version may no longer be accessible. August 23, 2011 DRAFT

[1]  E.J. Candes,et al.  An Introduction To Compressive Sampling , 2008, IEEE Signal Processing Magazine.

[2]  Pierre Moulin,et al.  Analysis of Multiresolution Image Denoising Schemes Using Generalized Gaussian and Complexity Priors , 1999, IEEE Trans. Inf. Theory.

[3]  H. Vincent Poor,et al.  Fast adaptive variational sparse Bayesian learning with automatic relevance determination , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Michael E. Tipping,et al.  Fast Marginal Likelihood Maximisation for Sparse Bayesian Models , 2003 .

[5]  Sundeep Rangan,et al.  Necessary and Sufficient Conditions for Sparsity Pattern Recovery , 2008, IEEE Transactions on Information Theory.

[6]  David P. Wipf,et al.  Variational Bayesian Inference Techniques , 2010, IEEE Signal Processing Magazine.

[7]  Robert D. Nowak,et al.  Compressed Channel Sensing: A New Approach to Estimating Sparse Multipath Channels , 2010, Proceedings of the IEEE.

[8]  P. Moulin,et al.  Analysis of multiresolution image denoising schemes using generalized-Gaussian priors , 1998, Proceedings of the IEEE-SP International Symposium on Time-Frequency and Time-Scale Analysis (Cat. No.98TH8380).

[9]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[10]  Richard G. Baraniuk,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[11]  Stephen J. Wright,et al.  Sparse reconstruction by separable approximation , 2009, IEEE Trans. Signal Process..

[12]  Michael E. Tipping Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[13]  D.G. Tzikas,et al.  The variational approximation for Bayesian inference , 2008, IEEE Signal Processing Magazine.

[14]  David Wipf,et al.  Iterative Reweighted � � and � Methods for Finding Sparse Solutions , 2010 .

[15]  B. Jørgensen Statistical Properties of the Generalized Inverse Gaussian Distribution , 1981 .

[16]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[17]  Dmitriy Shutin,et al.  Sparse Variational Bayesian SAGE Algorithm With Application to the Estimation of Multipath Wireless Channels , 2011, IEEE Transactions on Signal Processing.

[18]  David P. Wipf,et al.  Iterative Reweighted 1 and 2 Methods for Finding Sparse Solutions , 2010, IEEE J. Sel. Top. Signal Process..

[19]  E. Stacy A Generalization of the Gamma Distribution , 1962 .

[20]  David P. Wipf,et al.  A New View of Automatic Relevance Determination , 2007, NIPS.

[21]  Mário A. T. Figueiredo Adaptive Sparseness for Supervised Learning , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Charles M. Bishop,et al.  Variational Message Passing , 2005, J. Mach. Learn. Res..

[23]  H. Vincent Poor,et al.  Fast Variational Sparse Bayesian Learning With Automatic Relevance Determination for Superimposed Signals , 2011, IEEE Transactions on Signal Processing.

[24]  A. Doucet,et al.  A Hierarchical Bayesian Framework for Constructing Sparsity-inducing Priors , 2010, 1009.1914.

[25]  Stephen J. Wright,et al.  Sparse Reconstruction by Separable Approximation , 2008, IEEE Transactions on Signal Processing.

[26]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[27]  Brendan J. Frey,et al.  Factor graphs and the sum-product algorithm , 2001, IEEE Trans. Inf. Theory.

[28]  Milton Abramowitz,et al.  Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables , 1964 .

[29]  Bhaskar D. Rao,et al.  Sparse Bayesian learning for basis selection , 2004, IEEE Transactions on Signal Processing.