Estimation in Dirichlet random effects models

We develop a new Gibbs sampler for a linear mixed model with a Dirich- let process random effect term, which is easily extended to a generalized linear mixed model with a probit link function. Our Gibbs sampler exploits the properties of the multinomial and Dirichlet distributions, and is shown to be an improvement, in terms of operator norm and efficiency, over other commonly used MCMC algorithms. We also investigate methods for the estimation of the precision parameter of the Dirichlet process, finding that maximum likelihood may not be desirable, but a posterior mode is a reasonable approach. Examples are given to show how these models perform on real data. Our results complement both the theoretical basis of the Dirichlet process nonparametric prior and the computational work that has been done to date.

[1]  J. Hobert,et al.  Convergence rates and asymptotic standard errors for Markov chain Monte Carlo algorithms for Bayesian probit regression , 2007 .

[2]  G. Casella,et al.  Sampling Schemes for Generalized Linear Dirichlet Random Effects Models , 2009 .

[3]  J. Sethuraman.  A CONSTRUCTIVE DEFINITION OF DIRICHLET PRIORS , 1991 .

[4]  J. Hobert,et al.  A theoretical comparison of the data augmentation, marginal augmentation and PX-DA algorithms , 2008, 0804.0671.

[5]  Jun S. Liu.  Nonparametric hierarchical Bayes via sequential imputations , 1996 .

[6]  N. Breslow,et al.  Approximate inference in generalized linear mixed models , 1993 .

[7]  S. Chib,et al.  Bayesian analysis of binary and polychotomous response data , 1993 .

[8]  E. M. Crowley.  Product Partition Models for Normal Means , 1997 .

[9]  Hani Doss.  Bayesian Nonparametric Estimation of the Median; Part II: Asymptotic Properties of the Estimates , 1985 .

[10]  K. Das,et al.  INFERENCE IN DIRICHLET PROCESS MIXED GENERALIZED LINEAR MODELS BY USING MONTE CARLO EM , 2004 .

[11]  L. Schwartz.  On Bayes procedures , 1965 .

[12]  M. West.  On scale mixtures of normal distributions , 1987 .

[13]  P. McCullagh,et al.  Stochastic classification models , 2007 .

[14]  K. Das,et al.  Semiparametric Analysis of Two‐Level Bivariate Binary Data , 2006, Biometrics.

[15]  J. Hartigan,et al.  Product Partition Models for Change Point Problems , 1992 .

[16]  T. Ferguson.  A Bayesian Analysis of Some Nonparametric Problems , 1973 .

[17]  S. Ghosal,et al.  2 The Dirichlet process , related priors and posterior asymptotics , 2009 .

[18]  Albert Y. Lo,et al.  On a Class of Bayesian Nonparametric Estimates: I. Density Estimates , 1984 .

[19]  R. M. Korwar,et al.  Contributions to the Theory of Dirichlet Processes , 1973 .

[20]  Antonietta Mira,et al.  Ordering and Improving the Performance of Monte Carlo Markov Chains , 2001 .

[21]  D. Blackwell,et al.  Ferguson Distributions Via Polya Urn Schemes , 1973 .

[22]  M. Escobar,et al.  Bayesian Density Estimation and Inference Using Mixtures , 1995 .

[23]  Radford M. Neal.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[24]  Antonietta Mira,et al.  Ordering Monte Carlo Markov Chains , 1999 .

[25]  G. Casella,et al.  Clustering using objective functions and stochastic search , 2008 .

[26]  Michael,et al.  On a Class of Bayesian Nonparametric Estimates : I . Density Estimates , 2008 .

[27]  G. Casella,et al.  Nonparametric Priors for Ordinal Bayesian Social Science Models: Specification and Estimation , 2009 .

[28]  S. MacEachern,et al.  Estimating mixture of dirichlet process models , 1998 .

[29]  G. Casella.  Empirical Bayes Gibbs sampling. , 2001, Biostatistics.

[30]  F. Quintana,et al.  Bayesian clustering and product partition models , 2003 .

[31]  J. Ghosh,et al.  Consistent semiparametric Bayesian inference about a location parameter , 1999 .

[32]  Hani Doss.  Bayesian Nonparametric Estimation of the Median; Part I: Computation of the Estimates , 1985 .

[33]  Bradley P. Carlin,et al.  Generalized Linear Models for Small-Area Estimation , 1998 .

[34]  S. Ghosal.  Bayesian Nonparametrics: The Dirichlet process, related priors and posterior asymptotics , 2010 .

[35]  D. Freedman,et al.  On the consistency of Bayes estimates , 1986 .

[36]  Hani Doss.  Bayesian Nonparametric Estimation for Incomplete Data Via Successive Substitution Sampling , 1994 .

[37]  C. Macdonald.  Casella , 2004, Tempo.

[38]  D. Burr,et al.  A Bayesian Semiparametric Model for Random-Effects Meta-Analysis , 2005 .

[39]  Peter McCullagh,et al.  Stochastic classification models , 2006 .

[40]  A. Gelfand,et al.  Dirichlet Process Mixed Generalized Linear Models , 1997 .

[41]  J. Pitman.  Some developments of the Blackwell-MacQueen urn scheme , 1996 .

[42]  M. Escobar,et al.  Markov Chain Sampling Methods for Dirichlet Process Mixture Models , 2000 .

[43]  Li Zhang,et al.  Modeling Unobserved Sources of Heterogeneity in Animal Abundance Using a Dirichlet Process Prior , 2008, Biometrics.