Statistical Power for Nonequivalent Pretest-Posttest Designs

Nonequivalent controlled pretest-posttest designs are central to evaluation science, yet no practical and unified approach for estimating power in the two most widely used analytic approaches to these designs exists. This article fills the gap by presenting and comparing useful, unified power formulas for ANCOVA and change-score analyses, indicating the implications of each on sample-size requirements. The authors close with practical recommendations for evaluators. Mathematical details and a simple spreadsheet approach are included in appendices.

[1]  H A Feldman,et al.  Cohort versus cross-sectional design in large field trials: precision, sample size, and a unifying model. , 1994, Statistics in medicine.

[2]  Christopher Winship,et al.  THE ESTIMATION OF CAUSAL EFFECTS FROM OBSERVATIONAL DATA , 1999 .

[3]  Subhash C. Ray,et al.  Selection biases in sociological data , 1982 .

[4]  Steven H. Cohen,et al.  A Reexamination of Selection and Growth Processes in the Nonequivalent Control Group Design , 1980 .

[5]  Christopher Winship,et al.  Models for Sample Selection Bias , 1992 .

[6]  L. Bickman,et al.  Handbook of Applied Social Research Methods , 1998 .

[7]  P. Holland.  Statistics and Causal Inference , 1985 .

[8]  P. Bonate.  Analysis of pretest-posttest designs , 2000 .

[9]  Mark W. Lipsey,et al.  Design Sensitivity: Statistical Power for Experimental Research. , 1989 .

[10]  J D Dawson,et al.  Sample size calculations based on slopes and other summary statistics. , 1998, Biometrics.

[11]  B. Rosner.  The analysis of longitudinal data in epidemiologic studies. , 1979, Journal of Chronic Diseases.

[12]  Paul D. Allison,et al.  Using Panel Data to Estimate the Effects of Events , 1994 .

[13]  J. Stanley.  Quasi-Experimentation , 1965, The School Review.

[14]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[15]  S J Pocock,et al.  Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design. , 1992, Statistics in medicine.

[16]  L. F. Carter,et al.  Sociological Methodology 1980. , 1981 .

[17]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[18]  D. Bloch,et al.  A simple method of sample size calculation for linear and logistic regression. , 1998, Statistics in medicine.

[19]  James D. Wright,et al.  Handbook of Survey Research. , 1985 .

[20]  L. S. Feldt.  A comparison of the precision of three experimental designs employing a concomitant variable , 1958 .

[21]  R. Berk.  An introduction to sample selection bias in sociological data. , 1983 .

[22]  E. Maris.  Covariance adjustment versus gain scores-revisited , 1998 .

[23]  R. Linn,et al.  A general linear model for studying growth. , 1970 .

[24]  F. Lord.  LARGE‐SAMPLE COVARIANCE ANALYSIS WHEN THE CONTROL VARIABLE IS FALLIBLE* , 1958 .

[25]  R. Kronmal,et al.  Analysing the relationship between change in a risk factor and risk of disease. , 1992, Statistics in medicine.

[26]  C. D. Litton,et al.  Comparative Statistical Inference. , 1975 .

[27]  S. Assmann,et al.  Subgroup analysis and other (mis)uses of baseline data in clinical trials , 2000, The Lancet.

[28]  S Senn,et al.  Methods for assessing difference between groups in change when initial measurement is subject to intra-individual variation. , 1994, Statistics in medicine.

[29]  George W. Bohrnstedt,et al.  Chapter 3 – Measurement1 , 1983 .

[30]  C. Achilles,et al.  Evaluation: A Systematic Approach , 1980 .

[31]  A Donner,et al.  Approaches to sample size estimation in the design of clinical trials--a review. , 1984, Statistics in medicine.

[32]  D. Rubin.  ASSIGNMENT TO TREATMENT GROUP ON THE BASIS OF A COVARIATE , 1976 .

[33]  D. A. Kenny,et al.  A quasi-experimental approach to assessing treatment effects in the nonequivalent control group design. , 1975 .

[34]  N L Geller,et al.  The analysis of multiple endpoints in clinical trials. , 1987, Biometrics.

[35]  P. Lachenbruch,et al.  Design Sensitivity: Statistical Power for Experimental Research. , 1989 .

[36]  S J Pocock,et al.  Randomized trials or observational tribulations? , 2000, The New England journal of medicine.

[37]  Philip K. Robins,et al.  Evaluating Program Evaluations: New Evidence on Commonly Used Nonexperimental Methods , 1995 .

[38]  R A Kronmal,et al.  The effects of measurement error in response variables and tests of association of explanatory variables in change models. , 1998, Statistics in medicine.

[39]  T. Shakespeare,et al.  Observational Studies , 2003 .

[40]  P. Allison.  CHANGE SCORES AS DEPENDENT VARIABLES IN REGRESSION ANALYSIS , 1990 .

[41]  Jeffery A. Burr,et al.  Chapter 1 – Change Measurement* , 1990 .

[42]  E. Foster,et al.  An Evaluator's Guide To Detecting Attrition Problems , 1996 .

[43]  S. Lipsitz,et al.  Sample Size Calculations for Non‐Randomized Studies , 1995 .

[44]  F. Lord.  A paradox in the interpretation of group comparisons. , 1967, Psychological bulletin.

[45]  Steven G. Self,et al.  Power/Sample Size Calculations for Generalized Linear Models , 1988 .

[46]  Edgar F. Borgatta,et al.  Sociological Methodology, 1969. , 1969 .