The Effects of Type I Error Rate and Power of the ANCOVA F Test and Selected Alternatives Under Nonnormality and Variance Heterogeneity

Abstract The authors sought to identify through Monte Carlo simulations those conditions for which analysis of covariance (ANCOVA) does not maintain adequate Type I error rates and power. The conditions that were manipulated included assumptions of normality and variance homogeneity, sample size, number of treatment groups, and strength of the covariate-dependent variable relationship. Alternative tests studied were Quade's procedure, Puri and Sen's solution, Burnett and Barr's rank difference scores, Conover and Iman's rank transformation test, Hettmansperger's procedure, and the Puri-Sen-Harwell-Serlin test. For balanced designs, the ANCOVA F test was robust and was often the most powerful test through all sample-size designs and distributional configurations. With unbalanced designs, with variance heterogeneity, and when the largest treatment-group variance was matched with the largest group sample size, the nonparametric alternatives generally outperformed the ANCOVA test. When sample size and variance ratio were inversely coupled, all tests became very liberal; no test maintained adequate control over Type I error.

[1]  S. Hora.  Statistical Inference Based on Ranks , 1986 .

[2]  Pranab Kumar Sen,et al.  Analysis of Covariance Based on General Rank Scores , 1969 .

[3]  Allen I. Fleishman.  A method for simulating non-normal distributions , 1978 .

[4]  M. Atiqullah.  The robustness of the covariance analysis of a one-way classification , 1964 .

[5]  R L Iman,et al.  Analysis of covariance using the rank transformation. , 1982, Biometrics.

[6]  Janet D. Elashoff,et al.  Analysis of Covariance: A Delicate Instrument1 , 1969 .

[7]  L. Low,et al.  The Analysis of Covariance and Alternatives , 1983 .

[8]  Michael R. Harwell,et al.  A Nonparametric Test Statistic for the General Linear Model , 1989 .

[9]  D. Barr,et al.  A Nonparametric Analogy of Analysis of Covariance , 1977 .

[10]  S. Seaman,et al.  Type I Error Probabilities and Power of the Rank and Parametric Ancova Procedures , 1985 .

[11]  R. Serlin,et al.  An empirical study of a proposed test of nonparametric analysis of covariance. , 1988 .

[12]  Scott E. Maxwell,et al.  Another look at ANCOVA versus blocking. , 1984 .

[13]  David C. Hoaglin,et al.  The Reporting of Computation-Based Results in Statistics , 1975 .

[14]  Dana Quade,et al.  Rank Analysis of Covariance , 1967 .

[15]  Louis Laurencelle,et al.  Analysis of Covariance: A Proposed Algorithm , 1993 .

[16]  Michael R. Harwell,et al.  An Investigation of the Effect of Nonlinearity of Regression on the ANCOVA F Test. , 1997 .

[17]  J. Algina,et al.  A Review of Nonparametric Alternatives To Analysis of Covariance , 1985 .

[18]  Eryl A. C. Shirley,et al.  A Distribution‐Free Method for Analysis of Covariance Based on Ranked Data , 1981 .

[19]  M. E. Muller,et al.  A Note on the Generation of Random Normal Deviates , 1958 .

[20]  James Algina,et al.  Parametric ANCOVA and the Rank Transform ANCOVA When the Data are Conditionally Non-Normal and Heteroscedastic , 1984 .

[21]  E. Rakow,et al.  Effects of Violations of Data Set Assumptions When Using the Analysis of Variance and Covariance with Unequal Group Sizes. , 1994 .