Bayesian Inference for Logistic Models Using Pólya–Gamma Latent Variables

We propose a new data-augmentation strategy for fully Bayesian inference in models with binomial likelihoods. The approach appeals to a new class of Pólya–Gamma distributions, which are constructed in detail. A variety of examples are presented to show the versatility of the method, including logistic regression, negative binomial regression, nonlinear mixed-effect models, and spatial models for count data. In each case, our data-augmentation strategy leads to simple, effective methods for posterior inference that (1) circumvent the need for analytic approximations, numerical integration, or Metropolis–Hastings; and (2) outperform other known data-augmentation strategies, both in ease of use and in computational efficiency. All methods, including an efficient sampler for the Pólya–Gamma distribution, are implemented in the R package BayesLogit. Supplementary materials for this article are available online.

[1]  Brian D. Ripley,et al.  Pattern Recognition and Neural Networks , 1996 .

[2]  J. Carlin.  Meta-analysis for 2 x 2 tables: a Bayesian approach. , 1992, Statistics in medicine.

[3]  Warren B. Powell,et al.  The Effect of Robust Decisions on the Cost of Uncertainty in Military Airlift Operations , 2011, TOMC.

[4]  M. G. Pittau,et al.  A weakly informative default prior distribution for logistic and other regression models , 2008, 0901.4011.

[5]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[6]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[7]  Sylvia Frühwirth-Schnatter,et al.  Auxiliary mixture sampling with applications to logistic models , 2007, Comput. Stat. Data Anal..

[8]  James G. Scott,et al.  Data augmentation for non-Gaussian regression models using variance-mean mixtures , 2011, 1103.5407.

[9]  Peter E. Rossi,et al.  Bayesian Statistics and Marketing , 2005 .

[10]  Nicholas G. Polson,et al.  Simulation-based Regularized Logistic Regression , 2010, 1005.3430.

[11]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[12]  M. Plummer,et al.  CODA: convergence diagnosis and output analysis for MCMC , 2006 .

[13]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[14]  M. West.  On scale mixtures of normal distributions , 1987 .

[15]  D. McFadden.  Conditional logit analysis of qualitative choice behavior , 1972 .

[16]  Simon Jackman,et al.  Bayesian Analysis for the Social Sciences , 2009 .

[17]  A. Dawid.  Some matrix-variate distribution theory: Notational considerations and a Bayesian application , 1981 .

[18]  David B. Dunson,et al.  Lognormal and Gamma Mixed Negative Binomial Regression , 2012, ICML.

[19]  M. West,et al.  Data augmentation in multi-way contingency tables with fixed marginal totals , 2006 .

[20]  Peter E. Rossi,et al.  Bayesian Statistics and Marketing: Rossi/Bayesian Statistics and Marketing , 2006 .

[21]  Sylvia Frühwirth-Schnatter,et al.  Efficient MCMC for Binomial Logit Models , 2013, TOMC.

[22]  Sylvia Fr.  Data Augmentation and MCMC for Binary and Multinomial Logit Models , 2010 .

[23]  S. Frühwirth-Schnatter,et al.  Data Augmentation and MCMC for Binary and Multinomial Logit Models , 2010 .

[24]  C. Holmes,et al.  Bayesian auxiliary variable models for binary and multinomial regression , 2006 .

[25]  Leonhard Held,et al.  Improved auxiliary mixture sampling for hierarchical models of non-Gaussian data , 2009, Stat. Comput..

[26]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[27]  J C Wakefield,et al.  Hierarchical models for multicentre binary response studies. , 1990, Statistics in medicine.

[28]  Dani Gamerman,et al.  Sampling from the posterior distribution in generalized linear mixed models , 1997, Stat. Comput..

[29]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[30]  Z. Ciesielski,et al.  First passage times and sojourn times for Brownian motion in space and the exact Hausdorff measure of the sample path , 1962 .

[31]  John D. Lafferty,et al.  A correlated topic model of Science , 2007, 0708.3601.

[32]  J. Pitman,et al.  Probability laws related to the Jacobi theta and Riemann zeta functions, and Brownian excursions , 1999, math/9912170.

[33]  James G. Scott,et al.  Local shrinkage rules, Lévy processes and regularized regression , 2010, 1010.3390.

[34]  D. Dunson,et al.  Bayesian Multivariate Logistic Regression , 2004, Biometrics.

[35]  Daniele Durante,et al.  Bayesian dynamic financial networks with time-varying predictors , 2014, 1403.2272.

[36]  Jerry Nedelman,et al.  Book review: “Bayesian Data Analysis,” Second Edition by A. Gelman, J.B. Carlin, H.S. Stern, and D.B. Rubin Chapman & Hall/CRC, 2004 , 2005, Comput. Stat..

[37]  O. Barndorff-Nielsen,et al.  Normal Variance-Mean Mixtures and z Distributions , 1982 .

[38]  Tom Leonard,et al.  Bayesian Estimation Methods for Two‐Way Contingency Tables , 1975 .

[39]  Luc Devroye,et al.  On exact simulation algorithms for some distributions related to Jacobi theta functions , 2009 .

[40]  Andrew D. Martin,et al.  MCMCpack: Markov chain Monte Carlo in R , 2011 .

[41]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[42]  Hee Min Choi,et al.  The Polya-Gamma Gibbs sampler for Bayesian logistic regression is uniformly ergodic , 2013 .