An Application of the What Works Clearinghouse Standards for Evaluating Single-Subject Research

The use of single-subject research in the development and evaluation of academic, psychological, and behavioral interventions has led to the experimental validation of an array of treatment options for a diverse set of educational challenges. However, the synthesis of these bodies of research has been the subject of considerable debate. In an effort to utilize the findings from studies using single-subject methodologies to identify effective educational practices, the What Works Clearinghouse developed a set of criteria to evaluate the strength of evidence for various strategies. In this article, an application of these standards is demonstrated using a body of self-management intervention studies drawn from a recently published systematic review. The utility of the standards for identifying evidence-based practices validated with single-subject research methods is discussed. In addition, a comparison of the What Works Clearinghouse procedures to previously developed methods for identifying evidence-based practices with single-subject research is provided, and implications for research and practice are described.

[1]  Kimberly J. Vannest,et al.  Effect Sizes in Single Case Research: How Large is Large? , 2005 .

[2]  Paul Mooney,et al.  Status of and Trends in Academic Intervention Research for Students with Emotional Disturbance , 2003 .

[3]  Mark T. Greenberg,et al.  The Study of Implementation: Current Findings From Effective Programs that Prevent Mental Disorders in School-Aged Children , 2000 .

[4]  Robert E. Slavin,et al.  Perspectives on Evidence-Based Research in Education—What Works? Issues in Synthesizing Educational Program Evaluations , 2008 .

[5]  Peter J. Alter,et al.  A Descriptive Analysis of Positive Behavioral Intervention Research With Young Children With Challenging Behavior , 2005 .

[6]  Donald M. Baer,et al.  The Graphic Analysis of Data , 1986 .

[7]  Richard I. Parker,et al.  Evaluating Single-Case Research Data: A Comparison of Seven Statistical Methods , 2003 .

[8]  Robert H. Horner,et al.  Single-Case Designs Technical Documentation. , 2010 .

[9]  M. Wolery,et al.  Subject Descriptions and Single-Subject Research , 1993, Journal of learning disabilities.

[10]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[11]  F. Gresham,et al.  Relevance of functional behavioral assessment research for school-based interventions and positive behavioral support. , 2004, Research in developmental disabilities.

[12]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[13]  W. Shadish,et al.  The state of the science in the meta-analysis of single-case experimental designs , 2008 .

[14]  Allison Gorman Faith,et al.  Meta-analysis of Single-Case Research , 2014 .

[15]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[16]  K J Ottenbacher,et al.  Interrater agreement of visual analysis in single-subject decisions: quantitative review and analysis. , 1993, American journal of mental retardation : AJMR.

[17]  P. Strain,et al.  Evidence-Based Practice in Early Intervention/Early Childhood Special Education: Single-Subject Design Research , 2002 .

[18]  M. Wolery,et al.  Describing Baseline Conditions: Suggestions for Study Reports , 2007 .

[19]  R. Gersten,et al.  Research in Special Education: Scientific Methods and Evidence-Based Practices , 2005 .

[20]  Nancy E. Marchand-Martella,et al.  The Effects of a Self-Management Program in Reducing Off-Task Behavior , 1999 .

[21]  F. Volkmar,et al.  Development of the Evaluative Method for Evaluating and Determining Evidence-Based Practices in Autism , 2008, Journal of autism and developmental disorders.

[22]  Daniel M. Maggin,et al.  A systematic evaluation of token economies as a classroom management tool for students with challenging behavior. , 2011, Journal of school psychology.

[23]  Craig H. Kennedy,et al.  Single-Case Designs for Educational Research , 2004 .

[24]  Michael E Kelley,et al.  Visual aids and structured criteria for improving visual inspection and interpretation of single-case designs. , 2003, Journal of applied behavior analysis.

[25]  Cora J. M. Maas,et al.  Robustness issues in multilevel regression analysis , 2004 .

[26]  David Rindskopf,et al.  Methods for evidence‐based practice: Quantitative synthesis of single‐subject designs , 2007 .

[27]  Leanne Togher,et al.  Rating the methodological quality of single-subject designs and n-of-1 trials: Introducing the Single-Case Experimental Design (SCED) Scale , 2008, Neuropsychological rehabilitation.

[28]  C. Sachse-Lee,et al.  A Meta-Analysis of Single-Subject-Design Intervention Research for Students with LD , 2000, Journal of learning disabilities.

[29]  B. Gorman,et al.  Design and Analysis of Single-Case Research , 1996 .

[30]  Christina A. Christie,et al.  What Counts as Credible Evidence in Applied Research and Evaluation Practice , 2008 .

[31]  C. Heriza,et al.  Single‐subject research design: recommendations for levels of evidence and quality rating , 2008, Developmental medicine and child neurology.

[32]  Ke-Hai Yuan,et al.  On normal theory based inference for multilevel models with distributional violations , 2002 .

[33]  K. Holm,et al.  Single subject research. , 1983, Nursing research.

[34]  Hariharan Swaminathan,et al.  A generalized least squares regression approach for computing effect sizes in single-case research: application examples. , 2011, Journal of school psychology.

[35]  Thomas E. Scruggs,et al.  The Quantitative Synthesis of Single-Subject Research , 1987 .

[36]  David L. Gast,et al.  Single subject research methodology in behavioral sciences , 2010 .

[37]  Susan A. Murphy,et al.  A Conceptual Framework for Adaptive Preventive Interventions , 2004, Prevention Science.

[38]  Kimberly J. Vannest,et al.  Percentage of All Non-Overlapping Data (PAND) , 2007 .

[39]  Sandra M. Chafouleas,et al.  Review and Analysis of Literature on Self-Management Interventions to Promote Appropriate Classroom Behaviors (1988-2008). , 2009 .

[40]  Daniel M. Maggin,et al.  A Quantitative Synthesis of Methodology in the Meta-Analysis of Single-Subject Research for Students with Disabilities: 1985–2009 , 2011 .

[41]  S. Natasha Beretvas,et al.  A review of meta-analyses of single-subject experimental designs: Methodological issues and practice , 2008 .

[42]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .