A New Bayesian Lasso.

Park and Casella (2008) provided the Bayesian lasso for linear models by assigning scale mixture of normal (SMN) priors on the parameters and independent exponential priors on their variances. In this paper, we propose an alternative Bayesian analysis of the lasso problem. A different hierarchical formulation of Bayesian lasso is introduced by utilizing the scale mixture of uniform (SMU) representation of the Laplace density. We consider a fully Bayesian treatment that leads to a new Gibbs sampler with tractable full conditional posterior distributions. Empirical results and real data analyses show that the new algorithm has good mixing property and performs comparably to the existing Bayesian method in terms of both prediction accuracy and variable selection. An ECM algorithm is provided to compute the MAP estimates of the parameters. Easy extension to general models is also briefly discussed.

[1]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[2]  Chenlei Leng,et al.  Bayesian adaptive Lasso , 2010, Annals of the Institute of Statistical Mathematics.

[3]  M. West.  On scale mixtures of normal distributions , 1987 .

[4]  Mário A. T. Figueiredo.  Adaptive Sparseness for Supervised Learning , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  P. Bühlmann,et al.  The group lasso for logistic regression , 2008 .

[6]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[7]  Trevor Hastie,et al.  Regularization Paths for Generalized Linear Models via Coordinate Descent. , 2010, Journal of statistical software.

[8]  Cun-Hui Zhang.  Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[9]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[10]  Chenlei Leng,et al.  Unified LASSO Estimation by Least Squares Approximation , 2007 .

[11]  Zhaohui Qin,et al.  Uniform scale mixture models with applications to Bayesian inference , 2003 .

[12]  Wai-Yin Wan,et al.  Bayesian Student-T Stochastic Volatility Models Via Scale Mixtures , 2009 .

[13]  Samiran Ghosh,et al.  On the grouped selection and model complexity of the adaptive elastic net , 2011, Stat. Comput..

[14]  T. Hastie,et al.  [A Statistical View of Some Chemometrics Regression Tools]: Discussion , 1993 .

[15]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[16]  T. Stamey,et al.  Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate. II. Radical prostatectomy treated patients. , 1989, The Journal of urology.

[17]  Cun-Hui Zhang,et al.  A group bridge approach for variable selection , 2009, Biometrika.

[18]  Stephen Walker.  On scale mixtures of uniform distributions and the latent weighted least squares method , 1997 .

[19]  Hua Zhou,et al.  Penalized Regression for Genome-Wide Association Screening of Sequence Data , 2011, Pacific Symposium on Biocomputing.

[20]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[21]  Chris Hans.  Bayesian lasso regression , 2009 .

[22]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[23]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[24]  Cheolwoo Park,et al.  Bridge regression: Adaptivity and group selection , 2011 .

[25]  K. Lange,et al.  Coordinate descent algorithms for lasso penalized regression , 2008, 0803.3876.

[26]  Stephen G. Walker,et al.  Uniform Scale Mixture Models with Applications to Variance Regression , 2000 .

[27]  N. L. Johnson,et al.  Continuous Univariate Distributions. , 1995 .

[28]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[29]  G. Casella,et al.  Penalized regression, standard errors, and Bayesian lassos , 2010 .

[30]  Chris Hans,et al.  Model uncertainty and variable selection in Bayesian lasso regression , 2010, Stat. Comput..

[31]  Xiaohui Chen,et al.  A Bayesian Lasso via reversible-jump MCMC , 2011, Signal Process..

[32]  Qing Li,et al.  The Bayesian elastic net , 2010 .

[33]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[34]  S. Ghosh,et al.  Efficient sampling methods for truncated multivariate normal and student-t distributions subject to linear inequality constraints , 2015 .

[35]  Bani K. Mallick,et al.  Gene selection using a two-level hierarchical Bayesian model , 2004, Bioinform..