On the Convergence of Monte Carlo Maximum Likelihood Calculations

SUMMARY Monte Carlo maximum likelihood for normalized families of distributions can be used for an extremely broad class of models. Given any family { he: 0 E 0 } of non-negative integrable functions, maximum likelihood estimates in the family obtained by normalizing the functions to integrate to 1 can be approximated by Monte Carlo simulation, the only regularity conditions being a compactification of the parameter space such that the evaluation maps 0 h0(x) remain continuous. Then with probability 1 the Monte Carlo approximant to the log-likelihood hypoconverges to the exact log-likelihood, its maximizer converges to the exact maximum likelihood estimate, approximations to profile likelihoods hypoconverge to the exact profile and level sets of the approximate likelihood (support regions) converge to the exact sets (in Painleve-Kuratowski set convergence). The same results hold when there are missing data if a Wald-type integrability condition is satisfied. Asymptotic normality of the Monte Carlo error and convergence of the Monte Carlo approximation to the observed Fisher information are also shown.

[1]  L. Tierney.  Markov Chains for Exploring Posterior Distributions , 1994 .

[2]  E A Thompson,et al.  Monte Carlo estimation of mixed models for large complex pedigrees. , 1994, Biometrics.

[3]  A. Gelfand,et al.  Maximum-likelihood estimation for constrained- or missing-data models , 1993 .

[4]  C. Geyer,et al.  Analysis of Relatedness in the California Condors, from DNA Fingerprints , 1993 .

[5]  Kung-Sik Chan.  Asymptotic behavior of the Gibbs sampler , 1993 .

[6]  N. Sheehan,et al.  On the irreducibility of a Markov chain defined on a space of genotype configurations by a sampling scheme. , 1993, Biometrics.

[7]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[8]  C. Geyer,et al.  Constrained Monte Carlo Maximum Likelihood for Dependent Data , 1992 .

[9]  Roger J.-B. Wets,et al.  A characterization of epi-convergence in terms of convergence of level sets , 1992 .

[10]  P. Green,et al.  Metropolis Methods, Gaussian Proposals and Antithetic Variables , 1992 .

[11]  Xiao-Li Meng,et al.  Using EM to Obtain Asymptotic Variance-Covariance Matrices: The SEM Algorithm , 1991 .

[12]  E A Thompson,et al.  Evaluation of likelihood ratios for complex genetic models. , 1991, IMA journal of mathematics applied in medicine and biology.

[13]  John Geweke,et al.  Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments , 1991 .

[14]  Charles J. Geyer,et al.  Reweighting Monte Carlo Mixtures , 1991 .

[15]  G. C. Wei,et al.  A Monte Carlo Implementation of the EM Algorithm and the Poor Man's Data Augmentation Algorithms , 1990 .

[16]  Charles J. Geyer,et al.  Likelihood and Exponential Families , 1990 .

[17]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[18]  R. R. Bahadur.  Some Limit Theorems in Statistics , 1987 .

[19]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[20]  S. Varadhan,et al.  Central limit theorem for additive functionals of reversible Markov processes and applications to simple exclusions , 1986 .

[21]  H. Attouch.  Variational convergence for functions and operators , 1984 .

[22]  E. Nummelin.  General irreducible Markov chains and non-negative operators: Embedded renewal processes , 1984 .

[23]  E. Nummelin.  General irreducible Markov chains and non-negative operators: List of symbols and notation , 1984 .

[24]  E. Nummelin.  General irreducible Markov chains and non-negative operators: Notes and comments , 1984 .

[25]  D. B. Preston.  Spectral Analysis and Time Series , 1983 .

[26]  T. Louis.  Finding the Observed Information Matrix When Using the EM Algorithm , 1982 .

[27]  W. K. Hastings,et al.  Monte Carlo Sampling Methods Using Markov Chains and Their Applications , 1970 .

[28]  K. Chung.  Markov Chains with Stationary Transition Probabilities , 1961 .

[29]  R. R. Bahadur.  Examples of inconsistency of maximum likelihood estimates , 1958 .

[30]  J. Kiefer,et al.  CONSISTENCY OF THE MAXIMUM LIKELIHOOD ESTIMATOR IN THE PRESENCE OF INFINITELY MANY INCIDENTAL PARAMETERS , 1956 .

[31]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[32]  A. Wald.  Note on the Consistency of the Maximum Likelihood Estimate , 1949 .