Applying mixed-effects modeling to single-subject designs: An introduction.

Behavior analysis and statistical inference have shared a conflicted relationship for over fifty years. However, a significant portion of this conflict is directed toward statistical tests (e.g., t-tests, ANOVA) that aggregate group and/or temporal variability into means and standard deviations and as a result remove much of the data important to behavior analysts. Mixed-effects modeling, a more recently developed statistical test, addresses many of the limitations of more basic tests by incorporating random effects. Random effects quantify individual subject variability without eliminating it from the model, hence producing a model that can predict both group and individual behavior. We present the results of a generalized linear mixed-effects model applied to single-subject data taken from Ackerlund Brandt, Dozier, Juanico, Laudont, & Mick, 2015, in which children chose from one of three reinforcers for completing a task. Results of the mixed-effects modeling are consistent with visual analyses and importantly provide a statistical framework to predict individual behavior without requiring aggregation. We conclude by discussing the implications of these results and provide recommendations for further integration of mixed-effects models in the analyses of single-subject designs.

[1]  M Perone,et al.  Statistical inference in behavior analysis: Experimental control is better , 1999, The Behavior analyst.

[2]  John M. Ferron,et al.  Multilevel models for multiple-baseline data: modeling across-participant variation in autocorrelation and residual variance , 2013, Behavior research methods.

[3]  Casper W. Berg,et al.  glmmTMB Balances Speed and Flexibility Among Packages for Zero-inflated Generalized Linear Mixed Modeling , 2017, R J..

[4]  Michael E. Young,et al.  Discounting: A practical guide to multilevel analysis of choice data. , 2018, Journal of the experimental analysis of behavior.

[5]  Jillian M. Rung,et al.  Discounting of qualitatively different delayed health outcomes in current and never smokers. , 2016, Experimental and clinical psychopharmacology.

[6]  John M Ferron,et al.  From a single-level analysis to a multilevel analysis of single-case experimental designs. , 2014, Journal of school psychology.

[7]  Mirjam Moerbeek,et al.  Multilevel Analysis : Techniques and Applications, Third Edition , 2017 .

[8]  Jessica F. Juanico,et al.  The value of choice as a reinforcer for typically developing children. , 2015, Journal of applied behavior analysis.

[9]  M N Branch Statistical inference in behavior analysis: Some things significance testing does and does not do , 1999, The Behavior analyst.

[10]  M. Boisgontier,et al.  The anova to mixed model transition , 2016, Neuroscience & Biobehavioral Reviews.

[11]  W. Shadish,et al.  Using generalized additive (mixed) models to analyze single case designs. , 2014, Journal of school psychology.

[12]  Per B. Brockhoff,et al.  lmerTest Package: Tests in Linear Mixed Effects Models , 2017 .

[13]  Peter C Austin,et al.  Estimating Multilevel Logistic Regression Models When the Number of Clusters is Low: A Comparison of Different Statistical Software Procedures , 2010, The international journal of biostatistics.

[14]  Lili Tian,et al.  A Comparison of the General Linear Mixed Model and Repeated Measures ANOVA Using a Dataset with Multiple Missing Data Points , 2004, Biological research for nursing.

[15]  James M. LeBreton,et al.  Answers to 20 Questions About Interrater Reliability and Interrater Agreement , 2008 .

[16]  F. Symons,et al.  A Survey Evaluation of the Reliability of Visual Inspection and Functional Analysis Graphs , 2008, Behavior modification.

[17]  Peter R. Killeen,et al.  Predict, Control, and Replicate to Understand: How Statistics Can Foster the Fundamental Goals of Science , 2018, Perspectives on Behavior Science.

[18]  D. K. Singh,et al.  WEB PLOT DIGITIZER SOFTWARE: CAN IT BE USED TO MEASURE NECK POSTURE IN CLINICAL PRACTICE? , 2018, Asian Journal of Pharmaceutical and Clinical Research.

[19]  Michael C. Frank,et al.  Estimating the reproducibility of psychological science , 2015, Science.

[20]  N A Ator,et al.  Statistical inference in behavior analysis: Environmental determinants? , 1999, The Behavior analyst.

[21]  A. Banerjee,et al.  Hypothesis testing, type I and type II errors , 2009, Industrial psychiatry journal.

[22]  Laura M. Stapleton,et al.  The Effect of Small Sample Size on Two-Level Model Estimates: A Review and Illustration , 2014, Educational Psychology Review.

[23]  Michael E. Young,et al.  Discounting: A practical guide to multilevel analysis of indifference data. , 2017, Journal of the experimental analysis of behavior.

[24]  Michael E Kelley,et al.  Visual aids and structured criteria for improving visual inspection and interpretation of single-case designs. , 2003, Journal of applied behavior analysis.

[25]  M. Fay,et al.  Wilcoxon-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. , 2010, Statistics surveys.

[26]  Mollie E. Brooks,et al.  Generalized linear mixed models: a practical guide for ecology and evolution. , 2009, Trends in ecology & evolution.

[27]  Statistical inference in behavior analysis: Friend or foe? , 1999, The Behavior analyst.

[28]  D. Lerman,et al.  It has been said that, "There are three degrees of falsehoods: lies, damn lies, and statistics". , 2014, Journal of school psychology.

[29]  Daniel D. Drevon,et al.  Intercoder Reliability and Validity of WebPlotDigitizer in Extracting Graphed Data , 2017, Behavior modification.

[30]  R. Lanfear,et al.  The Extent and Consequences of P-Hacking in Science , 2015, PLoS biology.

[31]  Gene S Fisch,et al.  Evaluating data from behavioral analysis: visual inspection or statistical models? , 2001, Behavioural Processes.

[32]  Derek D Reed,et al.  A Microsoft Excel® 2010 Based Tool for Calculating Interobserver Agreement , 2011, Behavior analysis in practice.

[33]  William R. Nugent,et al.  Integrating Single-Case and Group-Comparison Designs for Evaluation Research , 1996 .