Estimating across-trial variability parameters of the Diffusion Decision Model: Expert advice and recommendations

For many years the Diffusion Decision Model (DDM) has successfully accounted for behavioral data from a wide range of domains. Important contributors to the DDM’s success are the across-trial variability parameters, which allow the model to account for the various shapes of response time distributions encountered in practice. However, several researchers have pointed out that estimating the variability parameters can be a challenging task. Moreover, the numerous fitting methods for the DDM each come with their own associated problems and solutions. This often leaves users in a difficult position. In this collaborative project we invited researchers from the DDM community to apply their various fitting methods to simulated data and provide advice and expert guidance on estimating the DDM’s across-trial variability parameters using these methods. Our study establishes a comprehensive reference resource and describes methods that can help to overcome the challenges associated with estimating the DDM’s across-trial variability parameters.

[1]  Thomas V. Wiecki,et al.  fMRI and EEG Predictors of Dynamic Decision Parameters during Human Reinforcement Learning , 2015, The Journal of Neuroscience.

[2]  Andreas Voss,et al.  Fast-dm: A free program for efficient diffusion model analysis , 2007, Behavior research methods.

[3]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[4]  Andrew Heathcote,et al.  Dynamic models of choice , 2018, Behavior Research Methods.

[5]  J. Starns.  Using response time modeling to distinguish memory and decision processes in recognition and source tasks , 2014, Memory & Cognition.

[6]  Scott D. Brown,et al.  Diffusion Decision Model: Current Issues and History , 2016, Trends in Cognitive Sciences.

[7]  Eric-Jan Wagenmakers,et al.  An EZ-diffusion model for response time and accuracy , 2007, Psychonomic bulletin & review.

[8]  Joachim Vandekerckhove,et al.  Extending JAGS: A tutorial on adding custom distributions to JAGS (with a diffusion model example) , 2013, Behavior Research Methods.

[9]  A. Voss,et al.  Interpreting the parameters of the diffusion model: An empirical validation , 2004, Memory & cognition.

[10]  Birte U. Forstmann,et al.  On the efficiency of neurally-informed cognitive models to identify latent cognitive states , 2017 .

[11]  A. Gelman,et al.  A non-degenerate estimator for hierarchical variance parameters via penalized likelihood estimation , 2011 .

[12]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[13]  R. Ratcliff,et al.  Connectionist and diffusion models of reaction time. , 1999, Psychological review.

[14]  Thomas V. Wiecki,et al.  HDDM: Hierarchical Bayesian estimation of the Drift-Diffusion Model in Python , 2013, Front. Neuroinform..

[15]  M. Lee,et al.  Bayesian Cognitive Modeling: A Practical Course , 2014 .

[16]  Daragh E. Sibley,et al.  Responding to nonwords in the lexical decision task: Insights from the English Lexicon Project. , 2015, Journal of experimental psychology. Learning, memory, and cognition.

[17]  W. Batchelder.  Multinomial processing tree models and psychological assessment. , 1998 .

[18]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .

[19]  Philip L. Smith,et al.  The accumulator model of two-choice discrimination , 1988 .

[20]  Jonathon Love,et al.  A Bayesian approach for estimating the probability of trigger failures in the stop-signal paradigm , 2016, Behavior research methods.

[21]  Jeffrey N. Rouder,et al.  Modeling Response Times for Two-Choice Decisions , 1998 .

[22]  Henrik Singmann,et al.  MPTinR: Analysis of multinomial processing tree models in R , 2013, Behavior research methods.

[23]  A. Voss,et al.  Model Complexity in Diffusion Modeling: Benefits of Making the Model More Parsimonious , 2016, Front. Psychol..

[24]  A. Anastasi.  Individual differences. , 2020, Annual review of psychology.

[25]  B. Efron,et al.  Stein's Paradox in Statistics , 1977 .

[26]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[27]  I. J. Myung,et al.  Tutorial on maximum likelihood estimation , 2003 .

[28]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[29]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[30]  Daragh E. Sibley,et al.  Individual Differences in Visual Word Recognition: Insights from the English Lexicon Project , 2012 .

[31]  Roger Ratcliff,et al.  A diffusion model account of the lexical decision task. , 2004, Psychological review.

[32]  Radford M. Neal.  Slice Sampling , 2003, The Annals of Statistics.

[33]  Roger Ratcliff,et al.  The EZ diffusion method: Too EZ? , 2008, Psychonomic bulletin & review.

[34]  Klaus Oberauer,et al.  How to use the diffusion model: Parameter recovery of three methods: EZ, fast-dm, and DMAT , 2009 .

[35]  Roger Ratcliff,et al.  Individual Differences and Fitting Methods for the Two-Choice Diffusion Model of Decision Making. , 2015, Decision.

[36]  Philip L. Smith,et al.  Attention orienting and the time course of perceptual decisions: response time distributions with masked and unmasked displays , 2004, Vision Research.

[37]  E. Wagenmakers,et al.  Psychological interpretation of the ex-Gaussian and shifted Wald parameters: A diffusion model analysis , 2009, Psychonomic bulletin & review.

[38]  Adam N. Sanborn,et al.  Model evaluation using grouped or individual data , 2008, Psychonomic bulletin & review.

[39]  Ken Aho,et al.  Model selection for ecologists: the worldviews of AIC and BIC. , 2014, Ecology.

[40]  Roger Ratcliff,et al.  Separating Implicit from Explicit Retrieval Processes in Perceptual Identification , 1996, Consciousness and Cognition.

[41]  R. Ratcliff.  A diffusion model account of response time and accuracy in a brightness discrimination task: Fitting real data and failing to fit fake but plausible data , 2002, Psychonomic bulletin & review.

[42]  R. Ratcliff,et al.  Sequential Sampling Models in Cognitive Neuroscience: Advantages, Applications, and Extensions. , 2016, Annual review of psychology.

[43]  Sophia Rabe-Hesketh,et al.  A Nondegenerate Penalized Likelihood Estimator for Variance Parameters in Multilevel Models , 2013, Psychometrika.

[44]  Rebecca Treiman,et al.  The English Lexicon Project , 2007, Behavior research methods.

[45]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[46]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[47]  Christopher H. Achen.  Two-Step Hierarchical Estimation: Beyond Regression Analysis , 2005, Political Analysis.

[48]  J. Kruschke.  Doing Bayesian Data Analysis , 2010 .

[49]  A. Voss,et al.  Retest reliability of the parameters of the Ratcliff diffusion model , 2017, Psychological research.

[50]  Hans-Jochen Heinze,et al.  Brain Areas Consistently Linked to Individual Differences in Perceptual Decision-making in Younger as well as Older Adults before and after Training , 2011, Journal of Cognitive Neuroscience.

[51]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[52]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[53]  M. Kenward,et al.  An Introduction to the Bootstrap , 2007 .

[54]  Donald Laming,et al.  Information theory of choice-reaction times , 1968 .

[55]  J. Vandekerckhove,et al.  The EZ diffusion model provides a powerful test of simple empirical effects , 2016, Psychonomic bulletin & review.

[56]  R. Ratcliff,et al.  Memory bias for negative emotional words in recognition memory is driven by effects of category membership , 2014, Cognition & emotion.

[57]  Philip L. Smith,et al.  An integrated theory of attention and decision making in visual signal detection. , 2009, Psychological review.

[58]  Scott D. Brown,et al.  The simplest complete model of choice response time: Linear ballistic accumulation , 2008, Cognitive Psychology.

[59]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[60]  Joachim Vandekerckhove,et al.  A cognitive latent variable model for the simultaneous analysis of behavioral and personality data. , 2014 .

[61]  M. Lee,et al.  Hierarchical diffusion models for two-choice response times. , 2011, Psychological methods.

[62]  R. Ratcliff,et al.  Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability , 2002, Psychonomic bulletin & review.

[63]  S. Carey,et al.  How counting represents number: What children must learn and when they learn it , 2008, Cognition.

[64]  Roger Ratcliff,et al.  Modeling perceptual discrimination in dynamic noise: Time-changed diffusion and release from inhibition , 2014 .

[65]  Andrew Heathcote,et al.  Linear Deterministic Accumulator Models of Simple Choice , 2012, Front. Psychology.

[66]  Cajo J. F. ter Braak,et al.  A Markov Chain Monte Carlo version of the genetic algorithm Differential Evolution: easy Bayesian computing for real parameter spaces , 2006, Stat. Comput..

[67]  R. Ratcliff,et al.  A Diffusion Model Account of Criterion Shifts in the Lexical Decision Task. , 2008, Journal of memory and language.

[68]  Andreas Voss,et al.  Assessing cognitive processes with diffusion model analyses: a tutorial based on fast-dm-30 , 2015, Front. Psychol..

[69]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[70]  K. S. Brown,et al.  Statistical mechanical approaches to models with many poorly known parameters. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[71]  A. Gelman.  Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[72]  Michael J Kane,et al.  Drifting from slow to "D'oh!": working memory capacity and mind wandering predict extreme reaction times and executive control errors. , 2012, Journal of experimental psychology. Learning, memory, and cognition.

[73]  John C. Nash,et al.  Unifying Optimization Algorithms to Aid Software System Users: optimx for R , 2011 .

[74]  John Geweke,et al.  Evaluating the accuracy of sampling-based approaches to the calculation of posterior moments , 1991 .

[75]  Roger Ratcliff,et al.  Statistical mimicking of reaction time data: Single-process models, parameter variability, and mixtures , 1995, Psychonomic bulletin & review.

[76]  R. Ratcliff,et al.  Neural Representation of Task Difficulty and Decision Making during Perceptual Categorization: A Timing Diagram , 2006, The Journal of Neuroscience.

[77]  Brandon M. Turner,et al.  A method for efficiently sampling from distributions with correlated dimensions. , 2013, Psychological methods.

[78]  H. Jeffreys,et al.  Theory of probability , 1896 .

[79]  Andreas Voss,et al.  How many trials are required for parameter estimation in diffusion modeling? A comparison of different optimization criteria , 2016, Behavior Research Methods.

[80]  Andreas Voss,et al.  A fast numerical algorithm for the estimation of diffusion model parameters , 2008 .

[81]  Karl Christoph Klauer,et al.  Separating response-execution bias from decision bias: arguments for an additional parameter in Ratcliff's diffusion model. , 2010, The British journal of mathematical and statistical psychology.

[82]  Francis Tuerlinckx,et al.  Fitting the ratcliff diffusion model to experimental data , 2007, Psychonomic bulletin & review.

[83]  Roger Ratcliff,et al.  Validating the unequal-variance assumption in recognition memory using response time distributions instead of ROC functions: A diffusion model analysis. , 2014, Journal of memory and language.

[84]  A. McQuarrie,et al.  Regression and Time Series Model Selection , 1998 .