Fitting Mixed-Effects Models Using Efficient EM-Type Algorithms

Abstract In recent years numerous advances in EM methodology have led to algorithms which can be very efficient when compared with both their EM predecessors and other numerical methods (e.g., algorithms based on Newton—Raphson). This article combines several of these new methods to develop a set of mode-finding algorithms for the popular mixed-effects model which are both fast and more reliable than such standard algorithms as proc mixed in SAS. We present efficient algorithms for maximum likelihood (ML), restricted maximum likelihood (REML), and computing posterior modes with conjugate proper and improper priors. These algorithms are not only useful in their own right, but also illustrate how parameter expansion, conditional data augmentation, and the ECME algorithm can be used in conjunction to form efficient algorithms. In particular, we illustrate a difficulty in using the typically very efficient PXEM (parameter-expanded EM) for posterior calculations, but show how algorithms based on conditional da...

[1]  Xiao-Li Meng,et al.  The Art of Data Augmentation , 2001 .

[2]  Emin Özçag,et al.  On the distribution , 2000 .

[3]  J. Foulley,et al.  The PX-EM algorithm for fast stable fitting of Henderson's mixed model , 2000, Genetics Selection Evolution.

[4]  Jun S. Liu,et al.  Parameter Expansion for Data Augmentation , 1999 .

[5]  Xiao-Li Meng,et al.  Seeking efficient data augmentation schemes via conditional and marginal augmentation , 1999 .

[6]  D. Rubin,et al.  Parameter expansion to accelerate EM: The PX-EM algorithm , 1998 .

[7]  George Casella,et al.  Functional Compatibility, Markov Chains and Gibbs Sampling with Improper Posteriors , 1998 .

[8]  G. Casella,et al.  The Effect of Improper Priors on Gibbs Sampling in Hierarchical Linear Mixed Models , 1996 .

[9]  J. Foulley,et al.  Heterogeneous variances in Gaussian linear mixed models , 1995, Genetics Selection Evolution.

[10]  D. Rubin,et al.  The ECME algorithm: A simple extension of EM and ECM with faster monotone convergence , 1994 .

[11]  Alfred O. Hero,et al.  Space-alternating generalized expectation-maximization algorithm , 1994, IEEE Trans. Signal Process..

[12]  Xiao-Li Meng,et al.  On the global and componentwise rates of convergence of the EM algorithm , 1994 .

[13]  D. Gianola,et al.  Marginal inferences about variance components in a mixed linear model using Gibbs sampling , 1993, Genetics Selection Evolution.

[14]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[15]  D. Harville,et al.  Some new algorithms for computing restricted maximum likelihood estimates of variance components , 1991 .

[16]  S. E. Hills,et al.  Illustration of Bayesian Inference in Normal Data Models Using Gibbs Sampling , 1990 .

[17]  D. Bates,et al.  Newton-Raphson and EM Algorithms for Linear Mixed-Effects Models for Repeated-Measures Data , 1988 .

[18]  Subir Ghosh,et al.  Statistical Analysis With Missing Data , 1988 .

[19]  C. N. Morris,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[20]  N. Laird,et al.  Maximum likelihood computations with repeated measures: application of the EM algorithm , 1987 .

[21]  K. Meyer,et al.  Estimation of variance components: What is missing in the EM algorithm? , 1986 .

[22]  Arthur P. Dempster,et al.  Statistical and Computational Aspects of Mixed Model Analysis , 1984 .

[23]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[24]  J. Ware,et al.  Random-effects models for longitudinal data. , 1982, Biometrics.

[25]  Nan M. Laird,et al.  Computation of variance components using the em algorithm , 1982 .

[26]  I. Olkin,et al.  Inequalities: Theory of Majorization and Its Applications , 1980 .

[27]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[28]  D. Harville.  Maximum Likelihood Approaches to Variance Component Estimation and to Related Problems , 1977 .

[29]  A. Beaton.  THE USE OF SPECIAL MATRIX OPERATORS IN STATISTICAL CALCULUS , 1964 .

[30]  P. Hsu.  ON THE DISTRIBUTION OF ROOTS OF CERTAIN DETERMINANTAL EQUATIONS , 1939 .

[31]  R. Fisher.  THE STATISTICAL UTILIZATION OF MULTIPLE MEASUREMENTS , 1938 .

[32]  D. V. Dyk.  NESTING EM ALGORITHMS FOR COMPUTATIONAL EFFICIENCY , 2000 .

[33]  D. Rubin,et al.  ML ESTIMATION OF THE t DISTRIBUTION USING EM AND ITS EXTENSIONS, ECM AND ECME , 1999 .

[34]  Xiao-Li Meng,et al.  Fast EM‐type implementations for mixed effects models , 1998 .

[35]  Xiao-Li Meng,et al.  The EM Algorithm—an Old Folk‐song Sung to a Fast New Tune , 1997 .

[36]  M. Fiedler.  Bounds for the determinant of the sum of hermitian matrices , 1971 .

[37]  Alfred O. Hero,et al.  Ieee Transactions on Image Processing: to Appear Penalized Maximum-likelihood Image Reconstruction Using Space-alternating Generalized Em Algorithms , 2022 .