A Comparison of Data Analysis Strategies for Testing Omnibus Effects in Higher-Order Repeated Measures Designs

Methods for analyzing repeated measures data, in addition to the conventional and corrected degrees of freedom univariate and multivariate solutions, are presented in this review. These "newer" methods offer researchers either improved control over Type I errors and/or greater power to detect treatment effects when (a) certain assumptions are violated, and/or (b) missing data exists. In particular, Huynh's (1978) Improved General Approximate method, a multivariate Welch (1951)/James (1951)-type test, the mixedmodel approach (Littell, Milliken, Stroup, & Wolfinger, 1996) and Boik's (1997) empirical Bayes method are discussed. We review the literature regarding these procedures with respect to their robustness, ability to handle missing data, and availability of software to obtain numerical results.

[1]  H J Keselman,et al.  A generally robust approach to hypothesis testing in independent and correlated groups designs. , 2003, Psychophysiology.

[2]  B. L. Welch.  The generalisation of student's problems when several different population variances are involved. , 1947, Biometrika.

[3]  Russell D. Wolfinger,et al.  A comparison of two approaches for selecting covariance structures in the analysis of repeated measurements , 1998 .

[4]  H J Keselman,et al.  Testing treatment effects in repeated measures designs: trimmed means and bootstrapping. , 2000, The British journal of mathematical and statistical psychology.

[5]  B. Lecoutre.  A Correction for the ε̃ Approximate Test in Repeated Measures Designs With Two or More Independent Groups , 1991 .

[6]  David E. Booth,et al.  Analysis of Incomplete Multivariate Data , 2000, Technometrics.

[7]  Welch Bl.  THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED , 1947 .

[8]  Jorge L. Mendoza,et al.  A significance test for multisample sphericity , 1980 .

[9]  Nicole A. Lazar,et al.  Statistical Analysis With Missing Data , 2003, Technometrics.

[10]  Chester L. Olson,et al.  Comparative Robustness of Six Tests in Multivariate Analysis of Variance , 1974 .

[11]  K. Bailey,et al.  Estimation and comparison of changes in the presence of informative right censoring: conditional linear model. , 1989, Biometrics.

[12]  Huynh Huynh,et al.  Some approximate tests for repeated measurement designs , 1978 .

[13]  A simulation study of estimators for rates of change in longitudinal studies with attrition. , 1991, Statistics in medicine.

[14]  Jorge L. Mendoza,et al.  Analysis of repeated measurements. , 1979 .

[15]  H J Keselman,et al.  Mixed-model pairwise multiple comparisons of repeated measures means. , 2001, Psychological methods.

[16]  R. J. Boik.  Analysis of Repeated Measures Under Second-Stage Sphericity: An Empirical Bayes Approach , 1997 .

[17]  A Power Comparison of the Welch-James and Improved General Approximation Tests in the Split-Plot Design , 1998 .

[18]  James Algina,et al.  An improved general approximation test for the main effect in a split‐plot design , 1995 .

[19]  J. Overall,et al.  Estimating sample sizes for repeated measurement designs. , 1994, Controlled clinical trials.

[20]  Marija J. Norusis,et al.  SPSS for Windows, Advanced Statistics, release 6.0 , 1993 .

[21]  H. Keselman,et al.  An examination of the robustness of the empirical Bayes and other approaches for testing main and interaction effects in repeated measures designs. , 2000, The British journal of mathematical and statistical psychology.

[22]  H. Keselman,et al.  The analysis of repeated measurements: A quantitative research synthesis , 1996 .

[23]  B. L. Welch.  ON THE COMPARISON OF SEVERAL MEAN VALUES: AN ALTERNATIVE APPROACH , 1951 .

[24]  S. Geisser,et al.  On methods in the analysis of profile data , 1959 .

[25]  M. Kenward,et al.  Small sample inference for fixed effects from restricted maximum likelihood. , 1997, Biometrics.

[26]  Russell D. Wolfinger,et al.  The analysis of repeated measurements: a comparison of mixed-model satterthwaite f tests and a nonpooled adjusted degrees of freedom multivariate test , 1999 .

[27]  C. Hertzog,et al.  Repeated-measures analysis of variance in developmental research: selected issues. , 1985, Child development.

[28]  H. J. Keselman,et al.  Analysing unbalanced repeated measures designs , 1990 .

[29]  R. Jennrich,et al.  Unbalanced repeated-measures models with structured covariance matrices. , 1986, Biometrics.

[30]  Roger E. Kirk,et al.  Experimental design: Procedures for the behavioral sciences (3rd ed.). , 1995 .

[31]  Donald Hedeker,et al.  Application of random-efiects pattern-mixture models for miss-ing data in longitudinal studies , 1997 .

[32]  H. Huynh,et al.  Conditions under Which Mean Square Ratios in Repeated Measurements Designs Have Exact F-Distributions , 1970 .

[33]  R. Kirk.  Experimental Design: Procedures for the Behavioral Sciences , 1970 .

[34]  R. R. Robey,et al.  Decisions in Single Group Repeated Measures Analysis: Statistical Tests and Three Computer Packages , 1984 .

[35]  Rand R. Wilcox,et al.  New designs in analysis of variance. , 1987 .

[36]  H. Rouanet,et al.  COMPARISON BETWEEN TREATMENTS IN A REPEATED‐MEASUREMENT DESIGN: ANOVA AND MULTIVARIATE METHODS , 1970 .

[37]  T. C. Oshima,et al.  Type I error rates for Huynh's general approximation and improved general approximation tests , 1994 .

[38]  Søren Johansen,et al.  Amendments and Corrections: The Welch--James Approximation to the Distribution of the Residual Sum of Squares in a Weighted Linear Regression , 1982 .

[39]  F B Baker,et al.  Estimates of test size for several test procedures based on conventional variance ratios in the repeated measures design , 1967, Psychometrika.

[40]  J. Richard Jennings,et al.  Editorial Policy on Analyses of Variance With Repeated Measures , 1987 .

[41]  Lisa M. Lix,et al.  Testing Repeated Measures Hypotheses When Covariance Matrices are Heterogeneous , 1993 .

[42]  H. Keselman,et al.  A comparison of recent approaches to the analysis of repeated measurements , 1999 .

[43]  L. K. Edwards,et al.  Applied analysis of variance in behavioral science , 1993 .

[44]  H. Akaike.  A new look at the statistical model identification , 1974 .

[45]  Bruno D. Zumbo,et al.  Investigation of the Robust Rank-Order Test for Non-Normal Populations with Unequal Variances: The Case of Reaction Time , 1997 .

[46]  Juliet Popper Shaffer,et al.  Multiple pairwise comparisons of repeated measures means under violation of multisample sphericity , 1991 .

[47]  H. Huynh,et al.  Estimation of the Box Correction for Degrees of Freedom from Sample Data in Randomized Block and Split-Plot Designs , 1976 .

[48]  THE ANALYSIS OF REPEATED MEASUREMENTS : UNIVARIATE TESTS, MULTIVARIATE TESTS, OR BOTH ? , 1995 .

[49]  D. Rubin.  INFERENCE AND MISSING DATA , 1975 .

[50]  Daniel R. Jeske,et al.  Mean Squared Error of Estimation or Prediction under a General Linear Model , 1992 .

[51]  H. Keselman,et al.  Interaction contrasts in repeated measures designs. , 1996 .

[52]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[53]  H. Keselman,et al.  Approximate degrees of freedom tests: A unified perspective on testing for mean equality. , 1995 .

[54]  H. Keselman,et al.  Comparing repeated measures means in factorial designs. , 1988, Psychophysiology.

[55]  G. S. James.  THE COMPARISON OF SEVERAL GROUPS OF OBSERVATIONS WHEN THE RATIOS OF THE POPULATION VARIANCES ARE UNKNOWN , 1951 .

[56]  Roderick J. A. Little,et al.  Modeling the Drop-Out Mechanism in Repeated-Measures Studies , 1995 .

[57]  Scott E. Maxwell,et al.  A Monte Carlo Comparison of Seven ε-Adjustment Procedures in Repeated Measures Designs With Small Sample Sizes , 1994 .

[58]  Generalization of Improved General Approximation tests to split‐plot designs with multiple between‐subjects factors and/or multiple within‐subjects factors , 1997 .

[59]  G. S. James.  TESTS OF LINEAR HYPOTHESES IN UNIVERIATE AND MULTIVARIATE ANALYSIS WHEN THE RATIOS OF THE POPULATION VARIANCES ARE UNKNOWN , 1954 .

[60]  Carl J. Huberty,et al.  Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses , 1998 .

[61]  J. Overall,et al.  Problematic formulations of SAS PROC.MIXED models for repeated measurements. , 1999, Journal of biopharmaceutical statistics.

[62]  Scott E. Maxwell,et al.  Designing Experiments and Analyzing Data: A Model Comparison Perspective , 1990 .

[63]  J. Miller,et al.  A warning about median reaction time. , 1988, Journal of experimental psychology. Human perception and performance.

[64]  Russell D. Wolfinger,et al.  Repeated Measures Analysis Using Mixed Models: Some Simulation Results , 1997 .

[65]  H. Keselman,et al.  Testing Repeated Measures Hypotheses when Covariance Matrices are Heterogeneous: Revisiting the Robustness of the Welch-James Test. , 1997, Multivariate behavioral research.

[66]  Some Alternative Approximate Tests for a Split Plot Design. , 1994, Multivariate behavioral research.

[67]  Mark Appelbaum,et al.  Bias in the Analysis of Repeated-Measures Designs: Some Alternative Approaches. , 1973 .

[68]  R. Wolfinger.  Heterogeneous Variance-Covariance Structures for Repeated Measures , 1996 .