Posterior consistency in linear models under shrinkage priors

We investigate the asymptotic behaviour of posterior distributions of regression coefficients in high-dimensional linear models as the number of dimensions grows with the number of observations. We show that the posterior distribution concentrates in neighbourhoods of the true parameter under simple sufficient conditions. These conditions hold under popular shrinkage priors given some sparsity assumptions. Copyright 2013, Oxford University Press.

[1]  Dominique Bontemps,et al.  Bernstein von Mises Theorems for Gaussian Regression with increasing number of regressors , 2010, 1009.1370.

[2]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[3]  Volkan Cevher,et al.  Learning with Compressible Priors , 2009, NIPS.

[4]  J. Bai,et al.  Multifactor asset pricing with a large number of observable risk factors and unobservable common and group-specific factors , 2013 .

[5]  Subhashis Ghosal,et al.  Asymptotic normality of posterior distributions in high-dimensional linear models , 1999 .

[6]  J. Griffin,et al.  Inference with normal-gamma prior distributions in regression problems , 2010 .

[7]  James G. Scott,et al.  The horseshoe estimator for sparse signals , 2010 .

[8]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[9]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[10]  Mário A. T. Figueiredo.  Adaptive Sparseness for Supervised Learning , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Michael E. Tipping.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[12]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[13]  M. Abramowitz,et al.  Handbook of Mathematical Functions With Formulas, Graphs and Mathematical Tables (National Bureau of Standards Applied Mathematics Series No. 55) , 1965 .

[14]  David B. Dunson,et al.  Generalized Beta Mixtures of Gaussians , 2011, NIPS.

[15]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[16]  P. Massart,et al.  Adaptive estimation of a quadratic functional by model selection , 2000 .

[17]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[18]  Christopher M. Bishop,et al.  Variational Relevance Vector Machines , 2000, UAI.

[19]  I. Johnstone,et al.  Needles and straw in haystacks: Empirical Bayes estimates of possibly sparse sequences , 2004, math/0410088.

[20]  D. Madigan,et al.  Bayesian Model Averaging for Linear Regression Models , 1997 .

[21]  Jaeyong Lee,et al.  GENERALIZED DOUBLE PARETO SHRINKAGE. , 2011, Statistica Sinica.

[22]  D. Owen.  Handbook of Mathematical Functions with Formulas , 1965 .

[23]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[24]  H. Zou,et al.  One-step Sparse Estimates in Nonconcave Penalized Likelihood Models. , 2008, Annals of statistics.

[25]  L. Schwartz.  On Bayes procedures , 1965 .

[26]  Wenxin Jiang.  Bayesian variable selection for high dimensional generalized linear models : Convergence rates of the fitted densities , 2007, 0710.3458.

[27]  Chris Hans.  Bayesian lasso regression , 2009 .

[28]  M. Clyde,et al.  Mixtures of g Priors for Bayesian Variable Selection , 2008 .

[29]  M. Yuan,et al.  Efficient Empirical Bayes Variable Selection and Estimation in Linear Models , 2005 .

[30]  Michael L. Littman,et al.  Bayesian Adaptive Sampling for Variable Selection and Model Averaging , 2011 .

[31]  Artin Armagan,et al.  Variational Bridge Regression , 2009, AISTATS.

[32]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[33]  J. Griffin,et al.  Bayesian adaptive lassos with non-convex penalization , 2007 .

[34]  Edward I. George,et al.  The Practical Implementation of Bayesian Model Selection , 2001 .