On multi-level modeling of data from repeated measures designs: a tutorial

Data from repeated measures experiments are usually analyzed with conventional ANOVA. Three well-known problems with ANOVA are the sphericity assumption, the design effect (sampling hierarchy), and the requirement for complete designs and data sets. This tutorial explains and demonstrates multi-level modeling (MLM) as an alternative analysis tool for repeated measures data. MLM allows us to estimate variance and covariance components explicitly. MLM does not require sphericity, it takes the sampling hierarchy into account, and it is capable of analyzing incomplete data. A fictitious data set is analyzed with MLM and ANOVA, and analysis results are compared. Moreover, existing data from a repeated measures design are re-analyzed with MLM, to demonstrate its advantages. Monte Carlo simulations suggest that MLM yields higher power than ANOVA, in particular under realistic circumstances. Although technically complex, MLM is recommended as a useful tool for analyzing repeated measures data from speech research.

[1]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[2]  E. A. Haggard.  Intraclass correlation and the analysis of variance. , 1960 .

[3]  Robert L. Bailey,et al.  Modeling and Prediction of Forest Growth Variables Based on Multilevel Nonlinear Mixed Models , 2001 .

[4]  S. Thompson,et al.  Multi-level models for repeated measurement data: application to quality of life data in clinical trials. , 1996, Statistics in medicine.

[5]  Judith D. Singer,et al.  Using SAS PROC MIXED to Fit Multilevel Models, Hierarchical Models, and Individual Growth Models , 1998 .

[6]  P. Onghena,et al.  Some issues in the statistical analysis of completely randomized and repeated measures designs for speech, language, and hearing research. , 1999, Journal of speech, language, and hearing research : JSLHR.

[7]  Roger E. Kirk,et al.  Experimental design: Procedures for the behavioral sciences (3rd ed.). , 1995 .

[8]  G. Parcel,et al.  Multilevel Models and Unbiased Tests for Group Based Interventions: Examples from the Safer Choices Study , 2001, Multivariate behavioral research.

[9]  R. Kirk.  Experimental Design: Procedures for the Behavioral Sciences , 1970 .

[10]  H. Goldstein.  Nonlinear multilevel models, with an application to discrete response data , 1991 .

[11]  Elazar J. Pedhazur,et al.  Measurement, Design, and Analysis: An Integrated Approach , 1994 .

[12]  Yuk Fai Cheong,et al.  HLM 6: Hierarchical Linear and Nonlinear Modeling , 2000 .

[13]  Kosuke Imai,et al.  Survey Sampling , 1998, Nov/Dec 2017.

[14]  Leo A. Goodman,et al.  A General Model for the Analysis of Surveys , 1972, American Journal of Sociology.

[15]  Rien van der Leeden,et al.  Multilevel Analysis of Repeated Measures Data , 1998 .

[16]  An application of multilevel analysis on writing process data , 1996 .

[17]  M K Kaiser,et al.  MANOVA method for analyzing repeated measures designs: an extensive primer. , 1985, Psychological bulletin.

[18]  A. Winsor.  Sampling techniques. , 2000, Nursing times.

[19]  Jan de Leeuw,et al.  Introducing Multilevel Modeling , 1998 .

[20]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[21]  L. Råstam,et al.  Diastolic blood pressure and area of residence: multilevel versus ecological analysis of social inequity , 2001, Journal of epidemiology and community health.

[22]  I. Kawachi,et al.  State-level income inequality and individual mortality risk: a prospective, multilevel study. , 2001, American journal of public health.

[23]  Mark Von Tress,et al.  Generalized, Linear, and Mixed Models , 2003, Technometrics.

[24]  Gert Rijlaarsdam,et al.  Importance in instructional text: teachers' and students' perceptions of task demands , 2002 .

[25]  V. Carey,et al.  Mixed-Effects Models in S and S-Plus , 2001 .

[26]  A. B.,et al.  SPEECH COMMUNICATION , 2001 .

[27]  Eva Nick,et al.  The dependability of behavioral measurements: theory of generalizability for scores and profiles , 1973 .

[28]  E. A. Haggard,et al.  Intraclass Correlation and the Analysis of Variance , 1958 .

[29]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[30]  J. Raaijmakers,et al.  How to deal with "The language-as-fixed-effect fallacy": Common misconceptions and alternative solutions. , 1999 .

[31]  Robert L. Mason,et al.  Statistical Principles in Experimental Design , 2003 .

[32]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data: A Model Comparison Perspective , 1990 .

[33]  Risto Lehtonen,et al.  Multilevel Statistical Models , 2005 .

[34]  William G. Cochran,et al.  Sampling Techniques, 3rd Edition , 1963 .

[35]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[36]  M. V. van Rossum,et al.  "Pitch" accent in alaryngeal speech. , 2002, Journal of speech, language, and hearing research : JSLHR.

[37]  M. Wade,et al.  On Indirect Genetic Effects in Structured Populations , 2001, The American Naturalist.

[38]  Harvey Goldstein,et al.  A general model for the analysis of multilevel data , 1988 .

[39]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[40]  B. J. Winer,et al.  Statistical Principles in Experimental Design, 2nd Edition. , 1973 .

[41]  Donald B. Rubin,et al.  The Dependability of Behavioral Measurements: Theory of Generalizability for Scores and Profiles. , 1974 .

[42]  Steven P. Reise,et al.  Introduction to the Special Issue on Multilevel Models , 2001, Multivariate behavioral research.

[43]  H Goldstein,et al.  Multilevel time series models with applications to repeated measures data. , 1994, Statistics in medicine.

[44]  Karl J. Friston,et al.  Variance Components , 2003 .

[45]  Ken Kelley,et al.  Designing Experiments and Analyzing Data: A Model Comparison Perspective, Third Edition , 1999 .

[46]  C. Michael Levy,et al.  The Science of Writing : Theories, Methods, Individual Differences and Applications , 1996 .

[47]  Joop J. Hox,et al.  Applied Multilevel Analysis. , 1995 .