An automated (Markov chain) Monte Carlo EM algorithm

We present an automated Monte Carlo EM (MCEM) algorithm which efficiently assesses Monte Carlo error in the presence of dependent Monte Carlo, particularly Markov chain Monte Carlo, E-step samples and chooses an appropriate Monte Carlo sample size to minimize this Monte Carlo error with respect to progressive EM step estimates. Monte Carlo error is gauged though an application of the central limit theorem during renewal periods of the MCMC sampler used in the E-step. The resulting normal approximation allows us to construct a rigorous and adaptive rule for updating the Monte Carlo sample size each iteration of the MCEM algorithm. We illustrate our automated routine and compare the performance with competing MCEM algorithms in an analysis of a data set fit by a generalized linear mixed model. †Supported by National Science Foundation/Environmental Protection Agency Grant DMS-99-78321 ‡E-mail: jjfan@sciences.sdsu.edu

[1]  Hani Doss.  Discussion: Markov Chains for Exploring Posterior Distributions , 1994 .

[2]  C. Robert,et al.  Convergence Controls for MCMC Algorithms with Applications to Hidden Markov Chains , 1999 .

[3]  C. McCulloch.  Maximum Likelihood Variance Components Estimation for Binary Data , 1994 .

[4]  C. Robert.  Simulation of truncated normal variables , 2009, 0907.4010.

[5]  Jun S. Liu,et al.  Monte Carlo EM with importance reweighting and its applications in random effects models 1 1 This wo , 1999 .

[6]  Andrew L. Rukhin,et al.  Tools for statistical inference , 1991 .

[7]  Bin Yu,et al.  Regeneration in Markov chain samplers , 1995 .

[8]  A. Kuk,et al.  MAXIMUM LIKELIHOOD ESTIMATION FOR PROBIT-LINEAR MIXED MODELS WITH CORRELATED RANDOM EFFECTS , 1997 .

[9]  George Casella,et al.  Implementations of the Monte Carlo EM Algorithm , 2001 .

[10]  J. H. Schuenemeyer,et al.  Generalized Linear Models (2nd ed.) , 1992 .

[11]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[12]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[13]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[14]  C. McCulloch.  Maximum Likelihood Algorithms for Generalized Linear Mixed Models , 1997 .

[15]  J. Booth,et al.  Maximizing generalized linear mixed model likelihoods with an automated Monte Carlo EM algorithm , 1999 .

[16]  C. Geyer,et al.  Discussion: Markov Chains for Exploring Posterior Distributions , 1994 .

[17]  D. V. Dyk.  NESTING EM ALGORITHMS FOR COMPUTATIONAL EFFICIENCY , 2000 .

[18]  Christian P. Robert,et al.  Monte Carlo Statistical Methods , 2005, Springer Texts in Statistics.