Determinantal Point Process Priors for Bayesian Variable Selection in Linear Regression

We propose discrete determinantal point processes (DPPs) for priors on the model parameter in Bayesian variable selection. By our variable selection method, collinear predictors are less likely to be selected simultaneously because of the repulsion property of discrete DPPs. Three types of DPP priors are proposed. We show the efficiency of the proposed priors through numerical experiments and applications to collinear datasets.

[1]  Brian J Reich,et al.  Consistent High-Dimensional Bayesian Variable Selection via Penalized Credible Regions , 2012, Journal of the American Statistical Association.

[2]  M. Yuan,et al.  Efficient Empirical Bayes Variable Selection and Estimation in Linear Models , 2005 .

[3]  E. Rains,et al.  Eynard–Mehta Theorem, Schur Process, and their Pfaffian Analogs , 2004, math-ph/0409059.

[4]  G. C. McDonald,et al.  Instabilities of Regression Estimates Relating Air Pollution to Mortality , 1973 .

[5]  A. E. Hoerl,et al.  Ridge regression: biased estimation for nonorthogonal problems , 2000 .

[6]  Veronika Rockova,et al.  EMVS: The EM Approach to Bayesian Variable Selection , 2014 .

[7]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[8]  Dean Phillips Foster,et al.  Calibration and Empirical Bayes Variable Selection , 1997 .

[9]  Yuval Peres,et al.  Zeros of Gaussian Analytic Functions and Determinantal Point Processes , 2009, University Lecture Series.

[10]  Marina Vannucci,et al.  An efficient stochastic search for Bayesian variable selection with high-dimensional correlated predictors , 2011, Comput. Stat. Data Anal..

[11]  Qing Li,et al.  The Bayesian elastic net , 2010 .

[12]  Ben Taskar,et al.  Determinantal Point Processes for Machine Learning , 2012, Found. Trends Mach. Learn..

[13]  O. Macchi.  The coincidence approach to stochastic point processes , 1975, Advances in Applied Probability.

[14]  David A. Belsley,et al.  Regression Analysis and its Application: A Data-Oriented Approach.@@@Applied Linear Regression.@@@Regression Diagnostics: Identifying Influential Data and Sources of Collinearity , 1981 .

[15]  W. W. Muir,et al.  Regression Diagnostics: Identifying Influential Data and Sources of Collinearity , 1980 .

[16]  Howard D Bondell,et al.  Bayesian variable selection using an adaptive powered correlation prior. , 2009, Journal of statistical planning and inference.

[17]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[18]  Dean P. Foster,et al.  The risk inflation criterion for multiple regression , 1994 .

[19]  C. Carvalho,et al.  Decoupling Shrinkage and Selection in Bayesian Linear Models: A Posterior Summary Perspective , 2014, 1408.0464.