Three-Level Analysis of Single-Case Experimental Data: Empirical Validation

One approach for combining single-case data involves use of multilevel modeling. In this article, the authors use a Monte Carlo simulation study to inform applied researchers under which realistic conditions the three-level model is appropriate. The authors vary the value of the immediate treatment effect and the treatment's effect on the time trend, the number of studies, cases and measurements, and the between-case and between-study variance. The study shows that the three-level approach results in unbiased estimates of both kinds of treatment effects. To have reasonable power for testing the treatment effects, the authors recommend researchers to use a homogeneous set of studies and to involve a minimum of 30 studies. The number of measurements and cases is of less importance.

[1]  Karrie A. Shogren,et al.  The Effect of Choice-Making as an Intervention for Problem Behavior , 2004 .

[2]  S. Natasha Beretvas,et al.  A review of meta-analyses of single-subject experimental designs: Methodological issues and practice , 2008 .

[3]  Joseph W. McKean,et al.  Two Reduced-Bias Autocorrelation Estimators: rF1 and rF2 , 1994 .

[4]  Jeffrey D. Kromrey,et al.  Determining the Efficacy of Intervention: The Use of Effect Sizes for Data Analysis in Single-Subject Research , 1996 .

[5]  Roel Bosker,et al.  Standard Errors and Sample Sizes for Two-Level Research , 1993 .

[6]  D. Barlow,et al.  Single Case Experimental Designs: Strategies for Studying Behavior Change , 1976 .

[7]  John Ferron,et al.  Statistical Power of Randomization Tests Used with Multiple-Baseline Designs , 2002 .

[8]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[9]  W. Shadish,et al.  Characteristics of single-case designs used to assess intervention effects in 2008 , 2011, Behavior research methods.

[10]  Russell D. Wolfinger,et al.  The Analysis of Repeated Measurements with Mixed-Model Adjusted F Tests , 2004 .

[11]  P. Onghena,et al.  Hierarchical linear models for the quantitative integration of effect sizes in single-case research , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[12]  F. G. Giesbrecht,et al.  Two-stage analysis based on a mixed model: large-sample asymptotic theory and small-sample simulation results , 1985 .

[13]  Samuel B. Green,et al.  The Impact of Misspecifying the Within-Subject Covariance Structure in Multiwave Longitudinal Multilevel Models: A Monte Carlo Study , 2007 .

[14]  R. N. Kackar,et al.  Approximations for Standard Errors of Estimators of Fixed and Random Effects in Mixed Linear Models , 1984 .

[15]  P. Onghena,et al.  Combining single-case experimental data using hierarchical linear models. , 2003 .

[16]  T Kratochwill,et al.  A further consideration in the application of an analysis-of-variance model for the intrasubject replication design. , 1974, Journal of applied behavior analysis.

[17]  Mary M. Kennedy,et al.  Generalizing From Single Case Studies , 1979 .

[18]  R. MacCoun.  Experimental and Quasi‐Experimental Designs for Generalized Causal Inference, by William R. Shadish, Thomas D. Cook, and Donald T. Campbell. Boston: Houghton Mifflin, 2001, 623 pp., $65.56. , 2003 .

[19]  Robin K. Morgan,et al.  Single-participant research design. Bringing science to managed care. , 2001, The American psychologist.

[20]  William R. Nugent,et al.  Integrating Single-Case and Group-Comparison Designs for Evaluation Research , 1996 .

[21]  Rachel T. Fouladi,et al.  A Comparison of Two General Approaches to Mixed Model Longitudinal Analyses Under Small Sample Size Conditions , 2004 .

[22]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[23]  J. McKean,et al.  A double bootstrap method to analyze linear models with autoregressive error terms. , 2000, Psychological methods.

[24]  John M Ferron,et al.  The Three-Level Synthesis of Standardized Single-Subject Experimental Data: A Monte Carlo Simulation Study , 2013, Multivariate behavioral research.

[25]  Patrick Onghena,et al.  The power of randomization tests for single-case phase designs , 1996 .

[26]  Susan T. Hibbard,et al.  Making treatment effect inferences from multiple-baseline data: The utility of multilevel modeling approaches , 2009, Behavior research methods.

[27]  Patrick Onghena,et al.  Customization of pain treatments: single-case design and analysis. , 2005, The Clinical journal of pain.

[28]  R. Parrila,et al.  Examining the effectiveness of peer-mediated and video-modeling social skills interventions for children with autism spectrum disorders: A meta-analysis in single-case research using HLM , 2011 .

[29]  Wim Van den Noortgate,et al.  Self-injurious behavior in people with profound intellectual disabilities: a meta-analysis of single-case studies. , 2011, Research in developmental disabilities.

[30]  Alan E. Kazdin,et al.  On resolving ambiguities of the multiple-baseline design: Problems and recommendations , 1975 .

[31]  Patrick Onghena,et al.  A multilevel meta-analysis of single-subject experimental design studies , 2008 .

[32]  L. Kern,et al.  Social Story™ Interventions for Students with Autism Spectrum Disorders: A Meta-Analysis , 2010, Journal of autism and developmental disorders.

[33]  Gilbert W. Fellingham,et al.  Performance of the Kenward–Roger Method when the Covariance Structure is Selected Using AIC and BIC , 2005 .

[34]  C. Sachse-Lee,et al.  A Meta-Analysis of Single-Subject-Design Intervention Research for Students with LD , 2000, Journal of learning disabilities.

[35]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[36]  Ann Casey,et al.  A Methodology for the Quantitative Synthesis of Intra-Subject Design Research , 1985 .

[37]  David Rindskopf,et al.  Methods for evidence‐based practice: Quantitative synthesis of single‐subject designs , 2007 .

[38]  Leanne Togher,et al.  Rating the methodological quality of single-subject designs and n-of-1 trials: Introducing the Single-Case Experimental Design (SCED) Scale , 2008, Neuropsychological rehabilitation.

[39]  John M Ferron,et al.  Estimating individual treatment effects from multiple-baseline data: A Monte Carlo study of multilevel-modeling approaches , 2010, Behavior research methods.

[40]  Bruce A. Thyer,et al.  Single-case experimental designs: Strategies for studying behavior change (2nd edn): by David H. Barlow and Michel Hersen Pergamon Press, New York, 1984, 419 pp. , 1985 .

[41]  John M Ferron,et al.  Synthesizing single-case studies: A Monte Carlo examination of a three-level meta-analytic model , 2012, Behavior research methods.

[42]  Keenan A. Pituch,et al.  The Performance of Multilevel Growth Curve Models Under an Autoregressive Moving Average Process , 2009 .

[43]  P. Cornelius,et al.  Approximate F-tests of multiple degree of freedom hypotheses in generalized least squares analyses of unbalanced split-plot experiments , 1996 .

[44]  F. E. Satterthwaite.  Synthesis of variance , 1941 .