Bayesian auxiliary variable models for binary and multinomial regression

In this paper we discuss auxiliary variable approaches to Bayesian binary and multinomial regression. These approaches are ideally suited to automated Markov chain Monte Carlo simulation. In the first part we describe a simple technique using joint updating that improves the performance of the conventional probit regression algorithm. In the second part we discuss auxiliary variable methods for inference in Bayesian logistic regression, including covariate set uncertainty. Finally, we show how the logistic method is easily extended to multinomial regression models. All of the algorithms are fully automatic with no user set parameters and no necessary Metropolis-Hastings accept/reject steps.

[1]  S. R. Searle,et al.  On Deriving the Inverse of a Sum of Matrices , 1981 .

[2]  L. Devroye.  Non-Uniform Random Variate Generation , 1986 .

[3]  M. West.  On scale mixtures of normal distributions , 1987 .

[4]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[5]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[6]  Adrian F. M. Smith,et al.  Bayesian Inference for Generalized Linear and Proportional Hazards Models Via Gibbs Sampling , 1993 .

[7]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[8]  David J. Spiegelhalter,et al.  Machine Learning, Neural and Statistical Classification , 2009 .

[9]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[10]  C. Robert.  Simulation of truncated normal variables , 2009, 0907.4010.

[11]  P. Green.  Reversible jump Markov chain Monte Carlo computation and Bayesian model determination , 1995 .

[12]  Dani Gamerman,et al.  Sampling from the posterior distribution in generalized linear mixed models , 1997, Stat. Comput..

[13]  D. K. Dey,et al.  BAYESIAN MODELING OF CORRELATED BINARY RESPONSES VIA SCALE MIXTURE OF MULTIVARIATE NORMAL LINK FUNCTIONS , 1998 .

[14]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo in Practice: A Roundtable Discussion , 1998 .

[15]  B. Mallick,et al.  Generalized Linear Models : A Bayesian Perspective , 2000 .

[16]  J H Albert,et al.  Sequential Ordinal Modeling with Applications to Survival Data , 2001, Biometrics.

[17]  Refik Soyer,et al.  Bayesian Methods for Nonlinear Classification and Regression , 2004, Technometrics.

[18]  Leonhard Held,et al.  Simultaneous Posterior Probability Statements From Monte Carlo Output , 2004 .