The Misuse of ANCOVA: The Academic and Political Implications of Type VI Errors In Studies of Achievement and Socioeconomic Status

California State University, Fresno University of Wisconsin, La Crosse University of Akron California State University, Fresno This paper examines ANCOVA designs which use SES as the covariate for achievement and Type VI errors. Type VI errors are inconsistencies between the research question and the research methodology, and these errors are discussed in the context of general semantics. The consequences of a Type VI error in studies of achievement differences covariating for SES can be highly misleading. When research with a Type VI error concludes that there are no significant differences in achievement across groups when statistically controlling for SES, the tacit implications are that actual achievement is consistent across groups and that SES can be causally controlled or is somehow not influential. Neither is correct. Authors suggest conducting validity studies of adjusted outcome scores to insure accuracy in interpreting results. he study of student achievement is a major focus of educational researchers and practitioners. With the recent passage of the No Student Left Behind legislation, the study of the factors related to increasing achievement scores has intensified. One salient variable which is correlated to achievement is SES (Attewell & Battle, 1999; Chapell & Overton, 2002; Gregory, 2000; O’Brien, Martinez-Pons & Kopala, 1999; Verna & Campbell, 1998), and researchers have used it as a covariate for achievement (Dillon & Schemo, 2004; Ferguson, 1981; Kaplan, 2002). The assumed reasoning is that if the variance attributable to SES is removed, the unique variance in achievement can be examined and explained. However, many errors in conceptualization and interpretation are possible with such designs. The purposes of this paper are: 1. To review the way in which ANCOVA is typically explained in textbooks commonly used in university statistics classes, 2. To provide a conceptual framework for discussions of numerical descriptions and the use of language, 3. To explain how Type VI errors, which are inconsistencies between the research question and the research methodology, can lead to inaccurate conclusions and/or interpretations of the data, 4. To provide descriptions of common errors in studies testing for group differences in achievement in which SES is used as the covariate, 5. To discuss the educational and political implications of such errors, and 6. To suggest that the adjusted scores in ANCOVA designs be correlated with other appropriate measures to determine their validity and correct interpretability.