A generalized least squares regression approach for computing effect sizes in single-case research: application examples.

A new method for deriving effect sizes from single-case designs is proposed. The strategy is applicable to small-sample time-series data with autoregressive errors. The method uses Generalized Least Squares (GLS) to model the autocorrelation of the data and estimate regression parameters to produce an effect size that represents the magnitude of treatment effect from baseline to treatment phases in standard deviation units. In this paper, the method is applied to two published examples using common single case designs (i.e., withdrawal and multiple-baseline). The results from these studies are described, and the method is compared to ten desirable criteria for single-case effect sizes. Based on the results of this application, we conclude with observations about the use of GLS as a support to visual analysis, provide recommendations for future research, and describe implications for practice.

[1]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[2]  F. Mosteller,et al.  Evidence matters : randomized trials in education research , 2002 .

[3]  R. Gersten,et al.  Research in Special Education: Scientific Methods and Evidence-Based Practices , 2005 .

[4]  Richard I. Parker,et al.  Useful effect size interpretations for single case research. , 2007, Behavior therapy.

[5]  M. Hunt.  How Science Takes Stock: The Story of Meta-Analysis , 1997 .

[6]  David Rindskopf,et al.  Methods for evidence‐based practice: Quantitative synthesis of single‐subject designs , 2007 .

[7]  P. Onghena,et al.  Hierarchical linear models for the quantitative integration of effect sizes in single-case research , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[8]  Ronald C. Serlin,et al.  Meta-analysis for single-case research. , 1992 .

[9]  Patrick Onghena,et al.  A multilevel meta-analysis of single-subject experimental design studies , 2008 .

[10]  R. Gorsuch.  Three methods for analyzing limited time-series (N of 1) data. , 1983 .

[11]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[12]  Robert M. Boody.  Scientific Research in Education: A Socratic Dialogue , 2011 .

[13]  K J Ottenbacher,et al.  Interrater agreement of visual analysis in single-subject decisions: quantitative review and analysis. , 1993, American journal of mental retardation : AJMR.

[14]  Bradley E. Huitema,et al.  Autocorrelation in applied behavior analysis: A myth. , 1985 .

[15]  Donald M. Baer,et al.  Meta-analysis for Single-Subject Research , 1987 .

[16]  K. Holm,et al.  Single subject research. , 1983, Nursing research.

[17]  W. Hays.  Single Subject Research in Special Education. , 1985 .

[18]  John M. Gottman,et al.  The Analysis of Change , 1995 .

[19]  D M Baer,et al.  Some current dimensions of applied behavior analysis. , 1968, Journal of applied behavior analysis.

[20]  Wallace J. Gingerich.  Meta-Analysis of Applied Time-Series Data , 1984 .

[21]  R. Shavelson,et al.  Scientific Research in Education , 2002 .

[22]  The Institute of Education Sciences: New Wine, New Bottles. , 2003 .

[23]  S. Natasha Beretvas,et al.  A review of meta-analyses of single-subject experimental designs: Methodological issues and practice , 2008 .

[24]  John M. Gottman,et al.  Time Series Analysis: A Comprehensive Introduction for Social Scientists. , 1983 .

[25]  Hsen-Hsing Ma.  An Alternative Method for Quantitative Synthesis of Single-Subject Researches , 2006, Behavior modification.

[26]  A. Deprospero,et al.  Inconsistent visual analyses of intrasubject data. , 1979, Journal of applied behavior analysis.

[27]  W. Shadish,et al.  The state of the science in the meta-analysis of single-case experimental designs , 2008 .

[28]  B. Gorman,et al.  Design and Analysis of Single-Case Research , 1996 .

[29]  Rumen Manolov,et al.  Comparing N = 1 Effect Size Indices in Presence of Autocorrelation , 2008, Behavior modification.

[30]  O. White,et al.  Some Comments Concerning “The Quantitative Synthesis of Single-Subject Research” , 1987 .

[31]  Roger P. Greenberg,et al.  How Science Takes Stock: The Story of Meta-Analysis. , 1998 .

[32]  J. Crosbie,et al.  Interrupted time-series analysis with brief single-subject data. , 1993, Journal of consulting and clinical psychology.

[33]  Leonard A. Marascuilo,et al.  Statistical analysis in single-case research: Issues, procedures, and recommendations, with applications to multiple behaviors. , 1992 .

[34]  Kimberly J. Vannest,et al.  Effect Sizes in Single Case Research: How Large is Large? , 2005 .

[35]  Erin E. Barton,et al.  Comparison of Overlap Methods for Quantitatively Synthesizing Single-Subject Data , 2010 .

[36]  Craig H. Kennedy,et al.  Single-Case Designs for Educational Research , 2004 .

[37]  D. Barlow,et al.  Single Case Experimental Designs: Strategies for Studying Behavior Change , 1976 .

[38]  B. E. Huitema.  Autocorrelation: 10 Years of confusion , 1988 .

[39]  Jonathan R. Flojo,et al.  Mathematics Instruction for Students With Learning Disabilities: A Meta-Analysis of Instructional Components , 2009 .

[40]  Robert H Horner,et al.  Bully prevention in positive behavior support. , 2009, Journal of applied behavior analysis.

[41]  Rumen Manolov,et al.  Lag-one autocorrelation in short series: Estimation and hypotheses testing , 2010 .

[42]  Michael P. Clements,et al.  FORECASTING ECONOMIC TIME SERIES , 2000, Econometric Theory.

[43]  Bradley E. Huitema.  Analysis of Interrupted Time-Series Experiments Using ITSE: A Critique , 2004 .

[44]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[45]  R. Kirk.  Practical Significance: A Concept Whose Time Has Come , 1996 .

[46]  Ann Casey,et al.  A Methodology for the Quantitative Synthesis of Intra-Subject Design Research , 1985 .

[47]  M. Wolery,et al.  The Use of Single-Subject Research to Identify Evidence-Based Practice in Special Education , 2005 .

[48]  B. Gorman,et al.  Calculating effect sizes for meta-analysis: the case of the single case. , 1993, Behaviour research and therapy.

[49]  The Comparative Analysis and Aggregation of Single-Case Data , 1988 .

[50]  W J Gingerich,et al.  Methodological observations on applied behavioral science. , 1984, The Journal of applied behavioral science.

[51]  J. McKean,et al.  A double bootstrap method to analyze linear models with autoregressive error terms. , 2000, Psychological methods.

[52]  Richard I. Parker,et al.  Controlling Baseline Trend in Single-Case Research. , 2006 .

[53]  R. Tervo,et al.  Single-Case Experimental Designs: Applications in Developmental-Behavioral Pediatrics , 2003, Journal of developmental and behavioral pediatrics : JDBP.

[54]  J. McKean,et al.  A Simple and Powerful Test for Autocorrelated Errors in OLS Intervention Models , 2000, Psychological reports.

[55]  D. Gast,et al.  Single Subject Research in Special Education , 1984 .

[56]  Richard I. Parker,et al.  Evaluating Single-Case Research Data: A Comparison of Seven Statistical Methods , 2003 .

[57]  Robert H. Horner,et al.  Single-Case Designs Technical Documentation. , 2010 .

[58]  D. Cochrane,et al.  Application of Least Squares Regression to Relationships Containing Auto-Correlated Error Terms , 1949 .

[59]  Kimberly J. Vannest,et al.  The Improvement Rate Difference for Single-Case Research , 2009 .

[60]  Thomas E. Scruggs,et al.  The Quantitative Synthesis of Single-Subject Research , 1987 .

[61]  Joseph W. McKean,et al.  Identifying Autocorrelation Generated by Various Error Processes in Interrupted Time-Series Regression Designs , 2007 .

[62]  B. S. Parsonson,et al.  2 – The Analysis and Presentation of Graphic Data , 1978 .

[63]  David L. Gast,et al.  Single subject research methodology in behavioral sciences , 2010 .

[64]  Alan Poling,et al.  Research methods in applied behavior analysis : issues and advances , 1986 .

[65]  William R. Jenson,et al.  Statistical reform: Evidence‐based practice, meta‐analyses, and single subject designs , 2007 .

[66]  James G. MacKinnon,et al.  A MAXIMUM LIKELIHOOD PROCEDURE FOR REGRESSION WITH AUTOCORRELATED ERRORS , 1978 .

[67]  B. Huitema.  Autocorrelation in Behavioral Research Wherefore Art Thou , 1986 .

[68]  William L. Heward,et al.  Effects of Response Cards on Disruptive Behavior and Academic Responding During Math Lessons by Fourth-Grade Urban Students , 2006 .

[69]  Kimberly J. Vannest,et al.  Percentage of All Non-Overlapping Data (PAND) , 2007 .

[70]  J. Levin,et al.  Single-case research design and analysis : new directions for psychology and education , 1992 .

[71]  H. L. Gray,et al.  Applied time series analysis , 2011 .