Model Misspecification and Assumption Violations With the Linear Mixed Model: A Meta-Analysis

This meta-analysis attempts to synthesize the Monte Carlo (MC) literature for the linear mixed model under a longitudinal framework. The meta-analysis aims to inform researchers about conditions that are important to consider when evaluating model assumptions and adequacy. In addition, the meta-analysis may be helpful to those wishing to design future MC simulations in identifying simulation conditions. The current meta-analysis will use the empirical type I error rate as the effect size and MC simulation conditions will be coded to serve as moderator variables. The type I error rate for the fixed and random effects will be explored as the primary dependent variable. Effect sizes were coded from 13 studies, resulting in a total of 4,002 and 621 effect sizes for fixed and random effects respectively. Meta-regression and proportional odds models were used to explore variation in the empirical type I error rate effect sizes. Implications for applied researchers and researchers planning new MC studies will be explored.

[1]  Cora J. M. Maas,et al.  The influence of violations of assumptions on multilevel parameter estimates and their standard errors , 2004, Comput. Stat. Data Anal..

[2]  J. Hammersley.  SIMULATION AND THE MONTE CARLO METHOD , 1982 .

[3]  John Ferron,et al.  Effects of Misspecifying the First-Level Error Structure in Two-Level Models of Change , 2002, Multivariate behavioral research.

[4]  J. Miller,et al.  The Inverse of the Freeman – Tukey Double Arcsine Transformation , 1978 .

[5]  J. Hox,et al.  Sufficient Sample Sizes for Multilevel Modeling , 2005 .

[6]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[7]  Keenan A. Pituch,et al.  The Performance of Multilevel Growth Curve Models Under an Autoregressive Moving Average Process , 2009 .

[8]  Samuel B. Green,et al.  The Impact of Misspecifying the Within-Subject Covariance Structure in Multiwave Longitudinal Multilevel Models: A Monte Carlo Study , 2007 .

[9]  Ofer Harel,et al.  Missing data techniques for multilevel data: implications of model misspecification , 2011 .

[10]  Michael R. Harwell,et al.  A multi-institutional study of high school mathematics curricula and college mathematics achievement and course taking , 2013 .

[11]  P. Diggle,et al.  Analysis of Longitudinal Data. , 1997 .

[12]  L. Hedges,et al.  The Handbook of Research Synthesis and Meta-Analysis , 2009 .

[13]  T. Yee.  The VGAM Package for Categorical Data Analysis , 2010 .

[14]  The Case for Use of Simple Difference Scores to Test the Significance of Differences in Mean Rates of Change in Controlled Repeated Measurements Designs , 2010, Multivariate behavioral research.

[15]  Harvey Goldstein,et al.  MCMC Sampling for a Multilevel Model With Nonindependent Residuals Within and Between Cluster Units , 2010 .

[16]  Thomas W. Yee,et al.  Vector Generalized Linear and Additive Models , 2015 .

[17]  Michael Frankfurter,et al.  Applied Longitudinal Data Analysis Modeling Change And Event Occurrence , 2016 .

[18]  J. Tukey,et al.  Transformations Related to the Angular and the Square Root , 1950 .

[19]  Kenneth A. Bollen,et al.  Monte Carlo Experiments: Design and Implementation , 2001 .

[20]  J. Ware,et al.  Applied Longitudinal Analysis , 2004 .

[21]  Wolfgang Viechtbauer,et al.  Conducting Meta-Analyses in R with the metafor Package , 2010 .

[22]  A Skrondal,et al.  Design and Analysis of Monte Carlo Experiments: Attacking the Conventional Wisdom , 2000, Multivariate Behavioral Research.

[23]  Thomas W. Yee,et al.  Vector Generalized Linear and Additive Models: With an Implementation in R , 2015 .

[24]  Harvey Goldstein,et al.  Multilevel Statistical Models: Goldstein/Multilevel Statistical Models , 2010 .

[25]  Harvey Goldstein,et al.  Likelihood methods for fitting multilevel models with complex level-1 variation , 2002 .

[26]  H. Goldstein.  Multilevel Statistical Models , 2006 .

[27]  William J. Browne,et al.  Implementation and performance issues in the Bayesian and likelihood fitting of multilevel models , 2000, Comput. Stat..

[28]  David C. Hoaglin,et al.  The Reporting of Computation-Based Results in Statistics , 1975 .

[29]  Cedric E. Ginestet.  ggplot2: Elegant Graphics for Data Analysis , 2011 .

[30]  J. Ware,et al.  Random-effects models for longitudinal data. , 1982, Biometrics.

[31]  Stephen W. Raudenbush,et al.  Analyzing effect sizes: Random-effects models. , 2009 .

[32]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[33]  Therese D. Pigott,et al.  An alternative to R2 for assessing linear models of effect size , 2010, Research synthesis methods.

[34]  Reuven Y. Rubinstein,et al.  Simulation and the Monte Carlo Method , 1981 .

[35]  Michael R. Harwell,et al.  Summarizing Monte Carlo Results in Methodological Research: The One- and Two-Factor Fixed Effects ANOVA Cases , 1992 .

[36]  Craig H. Mallinckrodt,et al.  Type I Error Rates from Mixed Effects Model Repeated Measures Versus Fixed Effects Anova with Missing Values Imputed Via Last Observation Carried Forward , 2001 .