Quality Appraisal of Single-Subject Experimental Designs: An Overview and Comparison of Different Appraisal Tools

Critical appraisal of the research literature is an essential step in informing and implementing evidence-based practice. Quality appraisal tools that assess the methodological quality of experimental studies provide a means to identify the most rigorous research suitable for evidence-based decision-making. In single-subject experimental research, quality appraisal is still in its infancy. Seven different quality appraisal tools were identified and compared with respect to their compliance with current standards for conducting single-subject experiments as well as their performance in evaluating research reports. Considerable variability was noted relative to the construction and content of the tools, which consequently led to variability in their evaluation results. Few tools provided empirical support for the validity of item construction and reliability of use. The Evaluative Method, the Certainty Framework, the What Works Clearinghouse Standards, and the Evidence in Augmentative and Alternative Communication Scales were identified as the more suitable instruments currently available for the critical appraisal of single-subject experimental designs, noting their different strengths and limitations. In the absence of a "gold standard critical appraisal tool," applied researchers and practitioners need to proceed with caution when interpreting evaluation results obtained from the existing tools, keeping their context and intent in mind.