Analyzing Single-Case Data: The Power of Randomization Tests.

Abstract Randomization tests have been proposed as a valid method for analyzing the data of single-case designs. In this study, the power of randomization tests was systematically examined for typical designs that rely on the random assignment of interventions within the sequence of observations. A 30-observation AB design, a 32-observation ABAB design, and a multiple-baseline AB design with 15 observations on each of four individuals were studied. Four levels of autocorrelation were considered, as well as six effect sizes, ranging from 0.0 to 1.4. For each combination of design, autocorrelation, and effect size, power was estimated by generating data for 1,000 experiments. The power estimates were generally found to be low.

[1]  Robert Gaylord-Ross,et al.  Visual Inspection and Statistical Analysis in Single-Case Designs , 1990 .

[2]  Bruce E. Wampold,et al.  Randomization tests for multiple-baseline designs. , 1986 .

[3]  C. Sharpley,et al.  Autocorrelation in behavioral data: An alternative perspective. , 1988 .

[4]  R. A. Fisher,et al.  Design of Experiments , 1936 .

[5]  Eugene S. Edgington,et al.  Overcoming Obstacles to Single-Subject Experimentation , 1980 .

[6]  Bruce E. Wampold,et al.  Randomization tests in single-subject designs: Illustrative examples , 1981 .

[7]  W. Velicer,et al.  A Comparison of Alternative Approaches to the Analysis of Interrupted Time-Series. , 1985, Multivariate behavioral research.

[8]  Kenneth M. Greenwood,et al.  Problems with the application of interrupted time series analysis for brief single-subject data , 1990 .

[9]  Dean Keith Simonton,et al.  Cross-sectional time-series experiments: Some suggested statistical analyses , 1977 .

[10]  Wayne F. Velicer,et al.  The Reliability and Accuracy of Time Series Model Identification , 1983 .

[11]  Eugene S. Edgington.  Nonparametric tests for single-case experiments. , 2015 .

[12]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[13]  Joseph W. McKean,et al.  Autocorrelation estimation and inference with small samples. , 1991 .

[14]  John M. Gottman,et al.  4 – Analysis of Interrupted Time-Series Experiments , 1978 .

[15]  Robert A. Stine,et al.  The Bias of Autoregressive Coefficient Estimators , 1988 .

[16]  B. Huitema.  Autocorrelation in Behavioral Research Wherefore Art Thou , 1986 .

[17]  Alan Poling,et al.  Research methods in applied behavior analysis : issues and advances , 1986 .

[18]  R. R. Jones,et al.  Effects of serial dependency on the agreement between visual and statistical inference. , 1978, Journal of applied behavior analysis.

[19]  Bradley E. Huitema,et al.  Autocorrelation in applied behavior analysis: A myth. , 1985 .

[20]  Thomas R. Kratochwill,et al.  Single Subject Research: Strategies for Evaluating Change , 1978 .

[21]  Donald M. Baer,et al.  The Graphic Analysis of Data , 1986 .

[22]  John M. Gottman,et al.  Time Series Analysis: A Comprehensive Introduction for Social Scientists. , 1983 .

[23]  E. Edgington.  Validity of Randomization Tests for one-Subject Experiments , 1980 .

[24]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[25]  Bradley E. Huitema.  Autocorrelation in Behavioral Research , 1986 .

[26]  B. E. Huitema.  Autocorrelation: 10 Years of confusion , 1988 .

[27]  T. Matyas,et al.  Visual analysis of single-case time series: Effects of variability, serial dependence, and magnitude of intervention effects. , 1990, Journal of applied behavior analysis.

[28]  K. J. Levy.  A Monte Carlo Study of a Method for Detecting a Change in the Slope of a Single Subject's Responses. , 1979 .

[29]  Alan E. Kazdin,et al.  Obstacles in Using Randomization Tests in Single-Case Experimentation , 1980 .

[30]  A. Deprospero,et al.  Inconsistent visual analyses of intrasubject data. , 1979, Journal of applied behavior analysis.

[31]  Larry E. Toothaker,et al.  N = 1 Designs: The Failure of ANOVA-Based Tests , 1983 .

[32]  Patrick Onghena,et al.  Randomization Tests for Extensions and Variations of ABAB Single-Case Experimental Designs: A Rejoinder , 1992 .

[33]  Phillip J. Brantley,et al.  Criterion-related validity of self-report measures of female heterosocial functioning , 1988 .

[34]  W F Velicer,et al.  Times Series Analysis Without Model Identification. , 1984, Multivariate behavioral research.

[35]  Thomas A. Matyas,et al.  Problems in the estimation of autocorrelation in brief time series and some implications for behavioral data , 1991 .

[36]  Joel R. Levin,et al.  3 – N = Nonparametric Randomization Tests , 1978 .

[37]  John Ferron.  Suggested Solutions to Problems Facing the Use of Randomization Tests with Single-Case Designs. , 1993 .

[38]  J Ferron,et al.  Using randomization tests with responsive single-case designs. , 1994, Behaviour research and therapy.

[39]  B. S. Parsonson,et al.  2 – The Analysis and Presentation of Graphic Data , 1978 .