Randomization procedures in single-case intervention research contexts: (Some of) "the rest of the story".

Following up on articles recently published in this journal, the present contribution tells (some of) "the rest of the story" about the value of randomization in single-case intervention research investigations. Invoking principles of internal, statistical-conclusion, and external validity, we begin by emphasizing the critical distinction between design randomization and analysis randomization, along with the necessary correspondence between the two. Four different types of single-case design-and-analysis randomization are then discussed. The persistent negative influence of serially dependent single-case outcome observations is highlighted, accompanied by examples of inappropriate applications of parametric and nonparametric tests that have appeared in the literature. We conclude by presenting valid applications of single-case randomization procedures in various single-case intervention contexts, with specific reference to a freely available Excel-based software package that can be accessed to incorporate the present randomization schemes into a wide variety of single-case intervention designs and analyses.

[1]  Matthew J. Koehler,et al.  Regulated Randomization: A Potentially Sharper Analytical Tool for the Multiple-Baseline Design , 1998 .

[2]  John M Ferron,et al.  Additional comparisons of randomization-test procedures for single-case multiple-baseline designs: Alternative effect types. , 2017, Journal of school psychology.

[3]  Bruce E. Wampold,et al.  Generalized Single-Case Randomization Tests: Flexible Analyses for a Variety of Situations. , 1999 .

[4]  M. Heyvaert,et al.  Randomization tests for single-case experiments: State of the art, state of the science, and state of the application , 2014 .

[5]  D. Elliffe,et al.  Rank-permutation tests for behavior analysis, and a test for trend allowing unequal data numbers for each subject. , 2019, Journal of the experimental analysis of behavior.

[6]  Joel R. Levin,et al.  Single-Case Intervention Research: Methodological and Statistical Advances , 2014 .

[7]  John Ferron,et al.  Statistical Power of Randomization Tests Used with Multiple-Baseline Designs , 2002 .

[8]  J. Levin,et al.  An Empirical Investigation of the Statistical Properties of Generalized Single-Case Randomization Tests. , 2004 .

[9]  Joel R. Levin,et al.  Randomized Classroom Trials on Trial , 2005 .

[10]  T. Pirinen,et al.  A single case study of a family-centred intervention with a young girl with cerebral palsy who is a multimodal communicator. , 2012, Child: care, health and development.

[11]  Kenneth W Jacobs,et al.  Replicability and randomization test logic in behavior analysis. , 2019, Journal of the experimental analysis of behavior.

[12]  J. Gentile,et al.  An analysis-of-variance model for the intrasubject replication design. , 1972, Journal of applied behavior analysis.

[13]  John M. Ferron,et al.  Comparison of randomization-test procedures for single-case multiple-baseline designs , 2018, Developmental neurorehabilitation.

[14]  John M Ferron,et al.  Nonparametric statistical tests for single-case systematic and randomized ABAB…AB and alternating treatment intervention designs: new developments, new directions. , 2012, Journal of school psychology.

[15]  Patrick Onghena,et al.  An R package for single-case randomization tests , 2008, Behavior research methods.

[16]  Robert H. Horner,et al.  Single-Case Intervention Research Design Standards , 2013 .

[17]  Patrick Onghena,et al.  Randomized single-case AB phase designs: Prospects and pitfalls , 2018, Behavior Research Methods.

[18]  John M. Ferron,et al.  Invited Article : Improved Randomization Tests for a Class of Single-Case Intervention Designs , 2015 .

[19]  E. Edgington.  Validity of Randomization Tests for one-Subject Experiments , 1980 .

[20]  Hyewon Chung,et al.  An evaluation of modified R 2-change effect size indices for single-subject experimental designs , 2008 .

[21]  Anya S. Evmenova,et al.  The single-case data-analysis ExPRT (Excel Package of Randomization Tests). , 2014 .

[22]  Eugene S. Edgington,et al.  Overcoming Obstacles to Single-Subject Experimentation , 1980 .

[23]  N. Lazar,et al.  The ASA Statement on p-Values: Context, Process, and Purpose , 2016 .

[24]  E S Edgington,et al.  Randomized single-subject experimental designs. , 1996, Behaviour research and therapy.

[25]  Jonathan J. Evans,et al.  The Single-Case Reporting Guideline In BEhavioural Interventions (SCRIBE) 2016 Statement , 2016, Neuropsychological rehabilitation.

[26]  John M. Ferron,et al.  Single-case permutation and randomization statistical tests: Present status, promising new developments. , 2014 .

[27]  W. Shadish,et al.  Characteristics of single-case designs used to assess intervention effects in 2008 , 2011, Behavior research methods.

[28]  Robert H. Horner,et al.  The What Works Clearinghouse Single-Case Design Pilot Standards , 2014 .

[29]  Andrew R Craig,et al.  Randomization tests as alternative analysis methods for behavior-analytic data. , 2019, Journal of the experimental analysis of behavior.

[30]  J. Levin,et al.  Pictorial mnemonic-strategy interventions for children with special needs: Illustration of a multiply randomized single-case crossover design , 2018, Developmental neurorehabilitation.

[31]  T Kratochwill,et al.  A further consideration in the application of an analysis-of-variance model for the intrasubject replication design. , 1974, Journal of applied behavior analysis.

[32]  Patrick Onghena,et al.  Randomization Tests for Extensions and Variations of ABAB Single-Case Experimental Designs: A Rejoinder , 1992 .

[33]  T. Rose,et al.  The functional relationship between artificial food colors and hyperactivity. , 1978, Journal of applied behavior analysis.

[34]  Joel R. Levin,et al.  3 – N = Nonparametric Randomization Tests , 1978 .

[35]  John Ferron.  Reconsidering the use of the general linear model with single-case data , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[36]  Justin D. Lane,et al.  Visual analysis in single case experimental design studies: Brief review and guidelines , 2014, Neuropsychological rehabilitation.

[37]  Carl J. Huberty,et al.  Statistical Practices of Educational Researchers: An Analysis of their ANOVA, MANOVA, and ANCOVA Analyses , 1998 .

[38]  J. Levin,et al.  Randomization Procedures for Changing Criterion Designs. , 2019, Behavior modification.

[39]  S. Revusky.  Some statistical treatments compatible with individual organism methodology. , 1967, Journal of the experimental analysis of behavior.

[40]  W. G. Cochran,et al.  Some consequences when the assumptions for the analysis of variance are not satisfied. , 1947, Biometrics.

[41]  Nirbhay N. Singh,et al.  A learning assessment procedure to re-evaluate three persons with a diagnosis of post-coma vegetative state and pervasive motor impairment , 2009, Brain injury.

[42]  D. James,et al.  Word-finding intervention for children with specific language impairment: a multiple single-case study. , 2012, Language, speech, and hearing services in schools.

[43]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[44]  J. Levin,et al.  Crafting educational intervention research that's both credible and creditable , 1994 .

[45]  Wim Van Den Noortgate,et al.  Permutation Tests in the Educational and Behavioral Sciences A Systematic Review , 2014 .

[46]  John Ferron,et al.  The Functioning of Single-Case Randomization Tests With and Without Random Assignment , 2003 .

[47]  N. Lazar,et al.  Moving to a World Beyond “p < 0.05” , 2019, The American Statistician.

[48]  Hariharan Swaminathan,et al.  A generalized least squares regression approach for computing effect sizes in single-case research: application examples. , 2011, Journal of school psychology.

[49]  J. Levin,et al.  Enhancing the scientific credibility of single-case intervention research: randomization to the rescue. , 2010, Psychological methods.

[50]  G. Glass,et al.  The Experimental Unit in Statistical Analysis , 1969 .

[51]  W. W. Rozeboom.  The fallacy of the null-hypothesis significance test. , 1960, Psychological bulletin.

[52]  Alan E. Kazdin,et al.  Obstacles in Using Randomization Tests in Single-Case Experimentation , 1980 .

[53]  E. Edgington.  Randomization Tests for One-Subject Operant Experiments , 1975 .

[54]  John M. Ferron,et al.  Analyzing Single-Case Data: The Power of Randomization Tests. , 1995 .