A Simple Sampler for the Horseshoe Estimator

In this note we derive a simple Bayesian sampler for linear regression with the horseshoe hierarchy. A new interpretation of the horseshoe model is presented, and extensions to logistic regression and alternative hierarchies, such as horseshoe+, are discussed. Due to the conjugacy of the proposed hierarchy, Chib's algorithm may be used to easily compute the marginal likelihood of the model.

[1]  S. Walker.  Invited comment on the paper "Slice Sampling" by Radford Neal , 2003 .

[2]  James G. Scott,et al.  The horseshoe estimator for sparse signals , 2010 .

[3]  James G. Scott,et al.  Shrink Globally, Act Locally: Sparse Bayesian Regularization and Prediction , 2022 .

[4]  W. Marsden.  I and J , 2012 .

[5]  H. Rue.  Fast sampling of Gaussian Markov random fields , 2000 .

[6]  James G. Scott,et al.  Local shrinkage rules, Lévy processes and regularized regression , 2010, 1010.3390.

[7]  Nicholas G. Polson,et al.  The Horseshoe+ Estimator of Ultra-Sparse Signals , 2015, 1502.00560.

[8]  D. Lindley,et al.  Bayes Estimates for the Linear Model , 1972 .

[9]  Bala Rajaratnam,et al.  Fast Bayesian Lasso for High-Dimensional Regression , 2015 .

[10]  James G. Scott,et al.  On the half-cauchy prior for a global scale parameter , 2011, 1104.4937.

[11]  James G. Scott,et al.  The Bayesian bridge , 2011, 1109.2279.

[12]  S. Chib.  Marginal Likelihood from the Gibbs Output , 1995 .

[13]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[14]  Hoon Kim,et al.  Monte Carlo Statistical Methods , 2000, Technometrics.

[15]  James G. Scott,et al.  Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables , 2012, 1205.0310.

[16]  B. Mallick,et al.  Fast sampling with Gaussian scale-mixture priors in high-dimensional regression. , 2015, Biometrika.

[17]  G. Casella,et al.  The Bayesian Lasso , 2008 .

[18]  Radford M. Neal.  Slice Sampling , 2003, The Annals of Statistics.

[19]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[20]  M. Wand,et al.  Mean field variational bayes for elaborate distributions , 2011 .

[21]  Nicholas G. Polson,et al.  Simulation-based Regularized Logistic Regression , 2010, 1005.3430.