Sparse Bayesian linear regression using generalized normal priors

A sparse Bayesian linear regression model is proposed that generalizes the Bayesian Lasso to a class of Bayesian models with scale mixtures of normal distributions as priors for the regression coefficients. We assume a hierarchical Bayesian model with a binary indicator for whether a predictor variable is included in the model, a generalized normal prior distribution for the coefficients of the included variables, and a Student-t error model for robustness to heavy tailed noise. Our model out-performs other popular sparse regression estimators on synthetic and real data.

[1]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[2]  R. Tibshirani,et al.  Sparsity and smoothness via the fused lasso , 2005 .

[3]  Michael E. Tipping.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[4]  Chris Hans,et al.  Model uncertainty and variable selection in Bayesian lasso regression , 2010, Stat. Comput..

[5]  Zongben Xu,et al.  The essential ability of sparse reconstruction of different compressive sensing strategies , 2011, Science China Information Sciences.

[6]  James G. Scott,et al.  The horseshoe estimator for sparse signals , 2010 .

[7]  A. Gelman.  Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[8]  Robert B. Gramacy,et al.  Shrinkage regression for multivariate inference with missing data, and an application to portfolio balancing , 2009, 0907.2135.

[9]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[10]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[11]  M. Steel,et al.  BAYESIAN REGRESSION ANALYSIS WITH SCALE MIXTURES OF NORMALS , 2000, Econometric Theory.

[12]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[13]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[14]  Stephen P. Boyd,et al.  Enhancing Sparsity by Reweighted ℓ1 Minimization , 2007, 0711.1612.

[15]  A. E. Hoerl,et al.  Ridge Regression: Applications to Nonorthogonal Problems , 1970 .

[16]  Mário A. T. Figueiredo.  Adaptive Sparseness for Supervised Learning , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Yufeng Liu,et al.  VARIABLE SELECTION IN QUANTILE REGRESSION , 2009 .

[18]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[19]  J. Griffin,et al.  Inference with normal-gamma prior distributions in regression problems , 2010 .

[20]  R. Tibshirani,et al.  Sparse inverse covariance estimation with the graphical lasso. , 2008, Biostatistics.

[21]  Xiaohui Chen,et al.  A Bayesian Lasso via reversible-jump MCMC , 2011, Signal Process..

[22]  Robert D. Nowak,et al.  Majorization–Minimization Algorithms for Wavelet-Based Image Restoration , 2007, IEEE Transactions on Image Processing.

[23]  Wang Yao,et al.  L 1/2 regularization , 2010 .

[24]  R. Tibshirani,et al.  PATHWISE COORDINATE OPTIMIZATION , 2007, 0708.1485.

[25]  Chris Hans.  Bayesian lasso regression , 2009 .

[26]  M. West.  On scale mixtures of normal distributions , 1987 .