Measurement-comparable effect sizes for single-case studies of free-operant behavior.

Single-case research comprises a set of designs and methods for evaluating the effects of interventions, practices, or programs on individual cases, through comparison of outcomes measured at different points in time. Although there has long been interest in meta-analytic techniques for synthesizing single-case research, there has been little scrutiny of whether proposed effect sizes remain on a directly comparable metric when outcomes are measured using different operational procedures. Much of single-case research focuses on behavioral outcomes in free-operant contexts, which may be measured using a variety of different direct observation procedures. This article describes a suite of effect sizes for quantifying changes in free-operant behavior, motivated by an alternating renewal process model that allows measurement comparability to be established in precise terms. These effect size metrics have the advantage of comporting with how direct observation data are actually collected and summarized. Effect size estimators are proposed that are applicable when the behavior being measured remains stable within a given treatment condition. The methods are illustrated by 2 examples, including a re-analysis of a systematic review of the effects of choice-making opportunities on problem behavior.

[1]  Kristine Jolivette,et al.  Effects of Choice-Making Opportunities on the Behavior of Students with Emotional and Behavioral Disorders , 2001 .

[2]  Ann Casey,et al.  A Methodology for the Quantitative Synthesis of Intra-Subject Design Research , 1985 .

[3]  S. Altmann,et al.  Estimating rates of behavior from Hansen frequencies , 1970, Primates.

[4]  Doug R. Moes,et al.  Integrating Choice-Making Opportunities within Teacher-Assigned Academic Tasks to Facilitate the Performance of Children with Autism , 1998 .

[5]  D. Gast,et al.  Dependent Measures and Measurement Procedures , 2009 .

[6]  Thomas E. Scruggs,et al.  PND at 25 , 2013 .

[7]  B. Gorman,et al.  Calculating effect sizes for meta-analysis: the case of the single case. , 1993, Behaviour research and therapy.

[8]  L. Bambara,et al.  Embedding Choice in the Context of Daily Routines: An Experimental Case Study , 1995 .

[9]  Jonathan J Deeks,et al.  Issues in the selection of a summary statistic for meta‐analysis of clinical trials with binary outcomes , 2002, Statistics in medicine.

[10]  R. Miltenberger,et al.  The influence of activity choice on problem behaviors maintained by escape versus attention. , 2002, Journal of applied behavior analysis.

[11]  Glen Dunlap,et al.  The effects of choice-making on the problem behaviors of high school students with intellectual disabilities , 1996 .

[12]  On the use of hansen frequencies for estimating rates of behavior , 1972, Primates.

[13]  G. Dunlap,et al.  Effects of choice making on the serious problem behaviors of students with severe handicaps. , 1990, Journal of applied behavior analysis.

[14]  W. Frea,et al.  A Demonstration of the Effects of Augmentative Communication on the Extreme Aggressive Behavior of a Child With Autism Within an Integrated Preschool Setting , 2001 .

[15]  Jessica Gurevitch,et al.  THE META‐ANALYSIS OF RESPONSE RATIOS IN EXPERIMENTAL ECOLOGY , 1999 .

[16]  M. Wolery,et al.  The Use of Single-Subject Research to Identify Evidence-Based Practice in Special Education , 2005 .

[17]  Alan E. Kazdin,et al.  International Handbook of Behavior Modification and Therapy , 1982 .

[18]  H. Pennypacker,et al.  Strategies and tactics of behavioral research, 2nd ed. , 1993 .

[19]  P. Onghena,et al.  Hierarchical linear models for the quantitative integration of effect sizes in single-case research , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[20]  David Rindskopf,et al.  Methods for evidence‐based practice: Quantitative synthesis of single‐subject designs , 2007 .

[21]  S. Natasha Beretvas,et al.  A review of meta-analyses of single-subject experimental designs: Methodological issues and practice , 2008 .

[22]  Alan E. Kazdin,et al.  Single-Case Research Designs: Methods for Clinical and Applied Settings , 2010 .

[23]  Kimberly J. Vannest,et al.  Combining nonoverlap and trend for single-case research: Tau-U. , 2011, Behavior therapy.

[24]  James E. Pustejovsky,et al.  A standardized mean difference effect size for multiple baseline designs across individuals , 2013, Research synthesis methods.

[25]  I Olkin,et al.  Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses. , 2000, Statistics in medicine.

[26]  Kimberly J. Vannest,et al.  Effect Size in Single-Case Research: A Review of Nine Nonoverlap Techniques , 2011, Behavior modification.

[27]  Eden Nagler Kyse,et al.  Analyzing data from single-case designs using multilevel models: new applications and some agenda items for future research. , 2013, Psychological methods.

[28]  Ronald C. Serlin,et al.  Meta-analysis for single-case research. , 1992 .

[29]  C. L. Cole,et al.  Effects of Within-Activity Choices on the Challenging Behavior of Children with Severe Developmental Disabilities , 2002 .

[30]  Daniel M. Maggin,et al.  A Quantitative Synthesis of Methodology in the Meta-Analysis of Single-Subject Research for Students with Disabilities: 1985–2009 , 2011 .

[31]  Erin E. Barton,et al.  Comparison of Overlap Methods for Quantitatively Synthesizing Single-Subject Data , 2010 .

[32]  Scott L. Hershberger,et al.  Meta-analysis of single-case designs , 1999 .

[33]  L. Hedges,et al.  Fixed- and random-effects models in meta-analysis. , 1998 .

[34]  Thomas R. Kratochwill,et al.  Evidence-based interventions in school psychology: Conceptual foundations of the Procedural and Coding Manual of Division 16 and the Society for the Study of School Psychology Task Force. , 2002 .

[35]  S. Lalley.  RENEWAL THEORY , 2014 .

[36]  David L. Gast,et al.  Applied Research in Education and Behavioral Sciences , 2014 .

[37]  Larry V. Hedges,et al.  What Are Effect Sizes and Why Do We Need Them , 2008 .

[38]  Levan Lim,et al.  Providing Choice Making Opportunities Within and Between Daily School Routines , 1999 .

[39]  Karrie A. Shogren,et al.  The Effect of Choice-Making as an Intervention for Problem Behavior , 2004 .

[40]  Jesse A. Berlin,et al.  Effect sizes for dichotomous data. , 2009 .

[41]  M. O'Reilly,et al.  A Review of School-Based Instructional Interventions for Students with Autism Spectrum Disorders. , 2008 .

[42]  Craig H. Kennedy,et al.  Single-Case Designs for Educational Research , 2004 .

[43]  R. Gersten,et al.  Research in Special Education: Scientific Methods and Evidence-Based Practices , 2005 .

[44]  W. N. Schoenfeld.  Problems of modern behavior theory , 1972, Conditional reflex.

[45]  Robert H. Hariharan George Keith Horner,et al.  Considerations for the Systematic Analysis and Use of Single-Case Research , 2012 .

[46]  David Rogosa,et al.  Statistical Models for Behavioral Observations , 1991 .

[47]  W J Gingerich,et al.  Methodological observations on applied behavioral science. , 1984, The Journal of applied behavioral science.

[48]  S. Disney,et al.  On the Lambert W function: EOQ applications and pedagogical considerations , 2010 .

[49]  Wolfgang Viechtbauer,et al.  Conducting Meta-Analyses in R with the metafor Package , 2010 .

[50]  Elizabeth Tipton,et al.  Erratum: Robust variance estimation in meta‐regression with dependent effect size estimates , 2010, Research synthesis methods.

[51]  J. Altmann,et al.  Observational study of behavior: sampling methods. , 1974, Behaviour.

[52]  Timothy J. Lewis,et al.  Functional Behavioral Assessment-Based Interventions for Students with or at Risk for Emotional and/or Behavioral Disorders in School: A Hierarchical Linear Modeling Meta-Analysis , 2012 .

[53]  W. Shadish,et al.  A standardized mean difference effect size for single case designs , 2012, Research synthesis methods.

[54]  G. Dunlap,et al.  Choice making to promote adaptive behavior for students with emotional and behavioral challenges. , 1994, Journal of applied behavior analysis.

[55]  Robert H Horner,et al.  Bully prevention in positive behavior support. , 2009, Journal of applied behavior analysis.

[56]  S Powell,et al.  Effects of choosing academic assignments on a student with attention deficit hyperactivity disorder. , 1997, Journal of applied behavior analysis.

[57]  Harris Cooper,et al.  HYPOTHESES AND PROBLEMS IN RESEARCH SYNTHESIS , 2019, The Handbook of Research Synthesis and Meta-Analysis.

[58]  Devender R. Banda,et al.  Picture Exchange Communication System With Individuals With Developmental Disabilities: A Meta-Analysis of Single Subject Studies , 2010 .

[59]  Thomas E. Scruggs,et al.  The Quantitative Synthesis of Single-Subject Research , 1987 .

[60]  Patrick Onghena,et al.  A multilevel meta-analysis of single-subject experimental design studies , 2008 .

[61]  Wallace J. Gingerich.  Meta-Analysis of Applied Time-Series Data , 1984 .

[62]  Jonathan M. Campbell,et al.  Statistics and Single Subject Research Methodology , 2009 .

[63]  L. Kern,et al.  Choice of Task Sequence to Reduce Problem Behaviors , 2001 .

[64]  V. Kulkarni.  Modeling and Analysis of Stochastic Systems , 1996 .

[65]  Hariharan Swaminathan,et al.  An effect size measure and Bayesian analysis of single-case designs. , 2014, Journal of school psychology.

[66]  Hariharan Swaminathan,et al.  A generalized least squares regression approach for computing effect sizes in single-case research: application examples. , 2011, Journal of school psychology.

[67]  J. Pustejovsky.  Operationally Comparable Effect Sizes for Meta-Analysis of Single-Case Research , 2013 .

[68]  T H Ollendick,et al.  Empirically supported psychological interventions: controversies and evidence. , 2001, Annual review of psychology.

[69]  Zili Sloboda,et al.  Defining and Measuring Drug Abusing Behaviors , 2005 .