Assessing Longitudinal Change: Adjustment for Regression to the Mean Effects

Pascarella (J Coll Stud Dev 47:508–520, 2006) has called for an increase in use of longitudinal data with pretest-posttest design when studying effects on college students. However, such designs that use multiple measures to document change are vulnerable to an important threat to internal validity, regression to the mean. Herein, we discuss a brief history of regression to the mean and illustrate a straightforward procedure to make adjustments to initial pretest scores for regression to the mean effects utilizing a method developed by Roberts (in: G. Echternacht (Guest ed.) New directions for testing and measurement, 1980). Analyses are shown with both unadjusted and adjusted pretest scores, illustrating dramatic differences in conclusions about whether students change across time.

[1]  J. L. Holland,et al.  Making vocational choices : a theory of vocational personalities and work environments , 1984 .

[2]  K. Schaie,et al.  Methodological issues in aging research , 1990 .

[3]  J. L. Holland,et al.  Making Vocational Choices: A Theory of Careers , 1973 .

[4]  A. O. H. Roberts.  Regression toward the Mean and the Regression-Effect Bias. , 1980 .

[5]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[6]  Ernest T. Pascarella.  How College Affects Students: Ten Directions for Future Research , 2006 .

[7]  Lita Furby,et al.  Interpreting regression toward the mean in developmental research. , 1973 .

[8]  A. Krause,et al.  Modeling and Simulation to Adjust p Values in Presence of a Regression to the Mean Effect , 2007 .

[9]  John L. Holland,et al.  The Psychology Of Vocational Choice , 1966 .

[10]  R. Reardon,et al.  Academic Disciplines: Holland's Theory and the Study of College Students and Faculty (review) , 2000 .

[11]  E. Reisner.  Assessment of the Title I Evaluation and Reporting System. , 1982 .

[12]  Frederic M. Lord.  The Measurement of Growth , 1956 .

[13]  David Rogosa,et al.  Myths about longitudinal research. , 1988 .

[14]  F. Galton.  Regression Towards Mediocrity in Hereditary Stature. , 1886 .

[15]  G. K. Tallmadge.  AN EMPIRICAL ASSESSMENT OF NORM-REFERENCED EVALUATION METHODOLOGY , 1982 .

[16]  R. Linn.  Discussion: Regression toward the Mean and the Interval between Test Administrations. , 1980 .

[17]  John C. Smart.  Attributes of Exemplary Research Manuscripts Employing Quantitative Analyses , 2005 .