High-Dimensional Posterior Consistency for Hierarchical Non-Local Priors in Regression

The choice of tuning parameter in Bayesian variable selection is a critical problem in modern statistics. Especially in the related work of nonlocal prior in regression setting, the scale parameter reflects the dispersion of the non-local prior density around zero, and implicitly determines the size of the regression coefficients that will be shrunk to zero. In this paper, we introduce a fully Bayesian approach with the pMOM nonlocal prior where we place an appropriate Inverse-Gamma prior on the tuning parameter to analyze a more robust model that is comparatively immune to misspecification of scale parameter. Under standard regularity assumptions, we extend the previous work where $p$ is bounded by the number of observations $n$ and establish strong model selection consistency when $p$ is allowed to increase at a polynomial rate with $n$. Through simulation studies, we demonstrate that our model selection procedure outperforms commonly used penalized likelihood methods in a range of simulation settings.

[1]  Valen E. Johnson,et al.  High-Dimensional Bayesian Classifiers Using Non-Local Priors , 2013, Statistical Models for Data Analysis.

[2]  Dean Phillips Foster,et al.  Calibration and empirical Bayes variable selection , 2000 .

[3]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[4]  F. Liang,et al.  Bayesian Subset Modeling for High-Dimensional Generalized Linear Models , 2013 .

[5]  Martin J. Wainwright,et al.  On the Computational Complexity of High-Dimensional Bayesian Variable Selection , 2015, ArXiv.

[6]  Brian J Reich,et al.  Consistent High-Dimensional Bayesian Variable Selection via Penalized Credible Regions , 2012, Journal of the American Statistical Association.

[7]  V. Johnson,et al.  On the use of non‐local prior densities in Bayesian hypothesis tests , 2010 .

[8]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[9]  Minsuk Shin,et al.  Scalable Bayesian Variable Selection Using Nonlocal Prior Densities in Ultrahigh-dimensional Settings. , 2015, Statistica Sinica.

[10]  Dean Phillips Foster,et al.  Calibration and Empirical Bayes Variable Selection , 1997 .

[11]  Cun-Hui Zhang.  Nearly unbiased variable selection under minimax concave penalty , 2010, 1002.4734.

[12]  Hongping Wu.  Nonlocal priors for Bayesian variable selection in generalized linear models and generalized linear mixed models and their applications in biology data , 2016 .

[13]  J. S. Rao,et al.  Spike and slab variable selection: Frequentist and Bayesian strategies , 2005, math/0505633.

[14]  Hongping Wu,et al.  A Note on Nonlocal Prior Method , 2017, 1702.07778.

[15]  M. Clyde,et al.  Mixtures of g Priors for Bayesian Variable Selection , 2008 .

[16]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[17]  N. Narisetty,et al.  Bayesian variable selection with shrinking and diffusing priors , 2014, 1405.6545.

[18]  F. Liang,et al.  High-Dimensional Variable Selection With Reciprocal L1-Regularization , 2015 .

[19]  Kshitij Khare,et al.  Posterior graph selection and estimation consistency for high-dimensional Bayesian DAG models , 2016, The Annals of Statistics.

[20]  V. Johnson,et al.  Bayesian Model Selection in High-Dimensional Settings , 2012, Journal of the American Statistical Association.

[21]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .