Learning Distributional Parameters for Adaptive Bayesian Sparse Signal Recovery

Power Exponential Scale Mixture (PESM), a generalized scale mixture family of distributions, has been recently proposed to model the sparsity inducing prior distributions currently in use for Sparse Signal Recovery (SSR). In this paper, we review this generalized scale mixture family and establish the necessary and sufficient condition for a distribution (symmetric with respect to origin) to have a PESM representation, which is a generalization of the results previously known for the Gaussian Scale Mixture (GSM) family. On the algorithmic front, we propose an adaptive Bayesian Sparse Signal Recovery (B-SSR) framework by learning the distributional parameters of a Generalized t-distribution (GT) which belongs to the PESM family. For specific choice of distributional parameters of GT our proposed framework corresponds to popular sparse recovery algorithms such as, LASSO, Reweighted ,1 norm minimization, Reweighted ,2 norm minimization etc. The tail nature of GT distribution family is extensively studied in this paper and an adaptive algorithm has been proposed where the tail nature of the prior is adapted over iterations based on the observation. Extensive experimental results based on traditional SSR setup have also been presented to show the efficacy of this adaptive approach.

[1]  Michael Elad,et al.  Sparse and Redundant Representations - From Theory to Applications in Signal and Image Processing , 2010 .

[2]  Wotao Yin,et al.  Iteratively reweighted algorithms for compressive sensing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  D. Rubin,et al.  ML ESTIMATION OF THE t DISTRIBUTION USING EM AND ITS EXTENSIONS, ECM AND ECME , 1999 .

[4]  Lawrence Carin,et al.  Bayesian Compressive Sensing , 2008, IEEE Transactions on Signal Processing.

[5]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[6]  J. Bernardo,et al.  Psi (Digamma) Function , 1976 .

[7]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[8]  Michael Elad,et al.  Optimally sparse representation in general (nonorthogonal) dictionaries via ℓ1 minimization , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[9]  J. McDonald,et al.  Skewness–kurtosis bounds for the skewed generalized T and related distributions , 2013 .

[10]  D. Donoho.  For most large underdetermined systems of equations, the minimal 𝓁1‐norm near‐solution approximates the sparsest near‐solution , 2006 .

[11]  Bruno A. Olshausen,et al.  Group Sparse Coding with a Laplacian Scale Mixture Prior , 2010, NIPS.

[12]  J. Keilson,et al.  MIXTURES OF DISTRIBUTIONS, MOMENT INEQUALITIES AND MEASURES OF EXPONENTIALITY AND NORMALITY' , 1974 .

[13]  David Vernon Widder,et al.  Laplace Transform (Pms-6) , 1942 .

[14]  P. Diaconis,et al.  Conjugate Priors for Exponential Families , 1979 .

[15]  Shinichi Nakajima,et al.  Bayesian Group-Sparse Modeling and Variational Inference , 2014, IEEE Transactions on Signal Processing.

[16]  René Vidal,et al.  Sparse subspace clustering , 2009, CVPR.

[17]  Bhaskar D. Rao,et al.  Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm , 1997, IEEE Trans. Signal Process..

[18]  Mike E. Davies,et al.  Latent Variable Analysis and Signal Separation , 2010 .

[19]  Richard J. Butler,et al.  Robust and Partially Adaptive Estimation of Regression Models , 1990 .

[20]  Richard G. Baraniuk,et al.  Bayesian Compressive Sensing Via Belief Propagation , 2008, IEEE Transactions on Signal Processing.

[21]  Kenneth Kreutz-Delgado,et al.  Strong Sub- and Super-Gaussianity , 2010, LVA/ICA.

[22]  Bhaskar D. Rao,et al.  Bootstrapped sparse Bayesian learning for sparse signal recovery , 2014, 2014 48th Asilomar Conference on Signals, Systems and Computers.

[23]  Michael Elad,et al.  Sparse Representation for Color Image Restoration , 2008, IEEE Transactions on Image Processing.

[24]  Balas K. Natarajan,et al.  Sparse Approximate Solutions to Linear Systems , 1995, SIAM J. Comput..

[25]  Bhaskar D. Rao,et al.  Perspectives on Sparse Bayesian Learning , 2003, NIPS.

[26]  D. Ruppert.  What is Kurtosis? An Influence Function Approach , 1987 .

[27]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[28]  Philip Schniter,et al.  Iteratively Reweighted 1 Approaches to Sparse , 2015 .

[29]  Gitta Kutyniok,et al.  1 . 2 Sparsity : A Reasonable Assumption ? , 2012 .

[30]  Bhaskar D. Rao,et al.  Variational EM Algorithms for Non-Gaussian Latent Variable Models , 2005, NIPS.

[31]  Bhaskar D. Rao,et al.  Type I and Type II Bayesian Methods for Sparse Signal Recovery Using Scale Mixtures , 2015, IEEE Transactions on Signal Processing.

[32]  Allen Y. Yang,et al.  Robust Face Recognition via Sparse Representation , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[34]  Robert D. Nowak,et al.  Majorization–Minimization Algorithms for Wavelet-Based Image Restoration , 2007, IEEE Transactions on Image Processing.

[35]  Bhaskar D. Rao,et al.  An Empirical Bayesian Strategy for Solving the Simultaneous Sparse Approximation Problem , 2007, IEEE Transactions on Signal Processing.

[36]  K. Lange,et al.  Normal/Independent Distributions and Their Applications in Robust Regression , 1993 .

[37]  Michael Elad,et al.  From Sparse Solutions of Systems of Equations to Sparse Modeling of Signals and Images , 2009, SIAM Rev..

[38]  Bhaskar D. Rao,et al.  Subset selection in noise based on diversity measure minimization , 2003, IEEE Trans. Signal Process..

[39]  J. Palmer.  Variational and scale mixture representations of non -Gaussian densities for estimation in the Bayesian linear model: Sparse coding, independent component analysis, and minimum entropy segmentation , 2006 .

[40]  George Eastman House,et al.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001 .

[41]  Allen Y. Yang,et al.  Distributed segmentation and classification of human actions using a wearable motion sensor network , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[42]  S. Bernstein,et al.  Sur les fonctions absolument monotones , 1929 .

[43]  Philip Schniter,et al.  An Empirical-Bayes Approach to Recovering Linearly Constrained Non-Negative Sparse Signals , 2014, IEEE Trans. Signal Process..

[44]  James B. McDonald,et al.  Partially Adaptive Estimation of Regression Models via the Generalized T Distribution , 1988, Econometric Theory.

[45]  Thomas Martinetz,et al.  Simple Method for High-Performance Digit Recognition Based on Sparse Coding , 2008, IEEE Transactions on Neural Networks.

[46]  David L Donoho,et al.  Compressed sensing , 2006, IEEE Transactions on Information Theory.

[47]  Aggelos K. Katsaggelos,et al.  Bayesian Compressive Sensing Using Laplace Priors , 2010, IEEE Transactions on Image Processing.

[48]  Bhaskar D. Rao,et al.  Latent Variable Bayesian Models for Promoting Sparsity , 2011, IEEE Transactions on Information Theory.

[49]  Bhaskar D. Rao,et al.  An affine scaling methodology for best basis selection , 1999, IEEE Trans. Signal Process..

[50]  I. Daubechies,et al.  Iteratively reweighted least squares minimization for sparse recovery , 2008, 0807.0575.

[51]  Lawrence Carin,et al.  Exploiting Structure in Wavelet-Based Bayesian Compressive Sensing , 2009, IEEE Transactions on Signal Processing.

[52]  Jaeyong Lee,et al.  GENERALIZED DOUBLE PARETO SHRINKAGE. , 2011, Statistica Sinica.