Model Choice and Diagnostics for Linear Mixed-Effects Models Using Statistics on Street Corners

ABSTRACT The complexity of linear mixed-effects (LME) models means that traditional diagnostics are rendered less effective. This is due to a breakdown of asymptotic results, boundary issues, and visible patterns in residual plots that are introduced by the model fitting process. Some of these issues are well known and adjustments have been proposed. Working with LME models typically requires that the analyst keeps track of all the special circumstances that may arise. In this article, we illustrate a simpler but generally applicable approach to diagnosing LME models. We explain how to use new visual inference methods for these purposes. The approach provides a unified framework for diagnosing LME fits and for model selection. We illustrate the use of this approach on several commonly available datasets. A large-scale Amazon Turk study was used to validate the methods. R code is provided for the analyses. Supplementary materials for this article are available online.

[1]  Heike Hofmann,et al.  Spatial Reasoning and Data Displays , 2016, IEEE Transactions on Visualization and Computer Graphics.

[2]  Andrew Pickles,et al.  Patterns of growth in verbal abilities among children with autism spectrum disorder. , 2007, Journal of consulting and clinical psychology.

[3]  Frederick Mosteller,et al.  A $k$-Sample Slippage Test for an Extreme Population , 1948 .

[4]  Tomasz Burzykowski,et al.  Linear Mixed Effects Model , 2021, Encyclopedia of Gerontology and Population Aging.

[5]  Heike Hofmann,et al.  Using visual statistical inference to better understand random class separations in high dimension, low sample size data , 2015, Comput. Stat..

[6]  Heike Hofmann,et al.  Graphical Tests for Power Comparison of Competing Designs , 2012, IEEE Transactions on Visualization and Computer Graphics.

[7]  E. Vonesh,et al.  Mixed-effects nonlinear regression for unbalanced repeated measures. , 1992, Biometrics.

[8]  Eugene Demidenko,et al.  Mixed Models: Theory and Applications with R , 2013 .

[9]  Deborah F. Swayne,et al.  Statistical inference for exploratory data analysis and model diagnostics , 2009, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[10]  C. Lord,et al.  Patterns of Growth in Adaptive Social Abilities Among Children with Autism Spectrum Disorders , 2009, Journal of abnormal child psychology.

[11]  L. Ryan,et al.  ASSESSING NORMALITY IN RANDOM EFFECTS MODELS , 1989 .

[12]  D. Stram,et al.  Variance components testing in the longitudinal mixed effects model. , 1994, Biometrics.

[13]  Robert Kosara,et al.  Do Mechanical Turks dream of square pie charts? , 2010, BELIV '10.

[14]  Gilbert W. Fellingham,et al.  Performance of the Kenward–Roger Method when the Covariance Structure is Selected Using AIC and BIC , 2005 .

[15]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .

[16]  J. Brian Gray,et al.  Applied Regression Including Computing and Graphics , 1999, Technometrics.

[17]  Heike Hofmann,et al.  Graphics of Large Datasets: Visualizing a Million , 2006 .

[18]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[19]  Heike Hofmann,et al.  Validation of Visual Statistical Inference, Applied to Linear Models , 2013 .

[20]  Andrew Gelman,et al.  Bayesian Measures of Explained Variance and Pooling in Multilevel (Hierarchical) Models , 2006, Technometrics.

[21]  H. Hofmann,et al.  Are You Normal? The Problem of Confounded Residual Structures in Hierarchical Linear Models , 2015 .

[22]  Adam Loy,et al.  HLMdiag: A Suite of Diagnostics for Hierarchical Linear Models in R , 2014 .

[23]  M. Kenward,et al.  Small sample inference for fixed effects from restricted maximum likelihood. , 1997, Biometrics.

[24]  Edsel A. Peña,et al.  Global Validation of Linear Model Assumptions , 2006, Journal of the American Statistical Association.

[25]  Heike Hofmann,et al.  Human Factors Influencing Visual Statistical Inference , 2014, 1408.1974.

[26]  Christopher H. Morrell,et al.  Lines in Random Effects Plots from the Linear Mixed-Effects Model , 2000 .

[27]  K E Muller,et al.  Tests for gaussian repeated measures with missing data in small samples. , 2000, Statistics in medicine.

[28]  Antony Unwin,et al.  Graphics of a Large Dataset , 2006 .

[29]  Adam Loy,et al.  Variations of Q–Q Plots: The Power of Our Eyes! , 2015, 1503.02098.

[30]  Ana Ivelisse Avilés,et al.  Linear Mixed Models for Longitudinal Data , 2001, Technometrics.

[31]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[32]  C. Morrell,et al.  Likelihood ratio testing of variance components in the linear mixed-effects model using restricted maximum likelihood. , 1998, Biometrics.

[33]  R. Carithers,et al.  Methylprednisolone therapy in patients with severe alcoholic hepatitis. A randomized multicenter trial. , 1989, Annals of internal medicine.

[34]  H. Pan,et al.  A Multilevel Analysis of School Examination Results , 1993 .

[35]  Jeffrey Heer,et al.  Crowdsourcing graphical perception: using mechanical turk to assess visualization design , 2010, CHI.

[36]  Michael G Kenward,et al.  The analysis of very small samples of repeated measurements I: An adjusted sandwich estimator , 2010, Statistics in medicine.

[37]  Jiming Jiang,et al.  ASYMPTOTIC PROPERTIES OF THE EMPIRICAL BLUP AND BLUE IN MIXED LINEAR MODELS , 1998 .

[38]  G Turk,et al.  The Mechanical Turk , 2015 .

[39]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[40]  Todd M. Gureckis,et al.  CUNY Academic , 2016 .

[41]  Hadley Wickham,et al.  ggplot2 - Elegant Graphics for Data Analysis (2nd Edition) , 2017 .