Speed–accuracy manipulations and diffusion modeling: Lack of discriminant validity of the manipulation or of the parameter estimates?

The diffusion model (Ratcliff, 1978) is a mathematical model theorized to untangle different cognitive processes involved in binary decision tasks. To test the validity of the diffusion model parameters, several experimental validation studies have been conducted. In these studies, the validity of the threshold separation parameter was tested with speed–accuracy manipulations. Typically, this manipulation not only results in the expected effect on the threshold separation parameter but it also impacts nondecision time: Nondecision time is longer in the accuracy than in the speed condition. There are two possible interpretations of the finding: On the one hand, it could indicate that speed versus accuracy instructions really have an impact on the duration of extradecisional processes. On the other hand, the effect on the measure for nondecision time could be spurious—that is, based on a problem in the parameter estimation procedures. In simulation studies—with the parameter sets based on typical values from experimental validation studies—we checked for possible biases in the parameter estimation. Our analyses strongly suggest that the observed pattern (i.e., slower nondecision processes under accuracy instructions) is attributable to a lack of discriminant validity of the manipulation rather than to trade-offs in the parameter estimations.

[1]  Andreas Voss,et al.  Fast-dm: A free program for efficient diffusion model analysis , 2007, Behavior research methods.

[2]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[3]  Michael F. Bunting,et al.  Alternatives to switch-cost scoring in the task-switching paradigm: Their reliability and increased validity , 2014, Behavior research methods.

[4]  A. Osman,et al.  Mechanisms of speed–accuracy tradeoff: evidence from covert motor processes , 2000, Biological Psychology.

[5]  R. Ratcliff,et al.  A diffusion model analysis of the effects of aging on letter discrimination. , 2003, Psychology and aging.

[6]  R. Ratcliff,et al.  The effects of aging on reaction time in a signal detection task. , 2001, Psychology and aging.

[7]  Andreas Voss,et al.  How many trials are required for parameter estimation in diffusion modeling? A comparison of different optimization criteria , 2016, Behavior Research Methods.

[8]  A. Voss,et al.  Interpreting the parameters of the diffusion model: An empirical validation , 2004, Memory & cognition.

[9]  Andreas Voss,et al.  Aging and emotional memory: cognitive mechanisms underlying the positivity effect. , 2008, Psychology and aging.

[10]  Francis Tuerlinckx,et al.  Diffusion model analysis with MATLAB: A DMAT primer , 2008, Behavior research methods.

[11]  Klaus Oberauer,et al.  How to use the diffusion model: Parameter recovery of three methods: EZ, fast-dm, and DMAT , 2009 .

[12]  A. Voss,et al.  Retest reliability of the parameters of the Ratcliff diffusion model , 2017, Psychological research.

[13]  Scott D. Brown,et al.  The simplest complete model of choice response time: Linear ballistic accumulation , 2008, Cognitive Psychology.

[14]  Richard P. Heitz,et al.  The speed-accuracy tradeoff: history, physiology, methodology, and behavior , 2014, Front. Neurosci..

[15]  Verena C. Seibold,et al.  Does temporal preparation increase the rate of sensory information accumulation? , 2011, Acta psychologica.

[16]  A. Osman,et al.  On the locus of speed-accuracy trade-off in reaction time: inferences from the lateralized readiness potential. , 2004, Journal of experimental psychology. General.

[17]  Klaus Oberauer,et al.  Individual differences in components of reaction time distributions and their relations to working memory and intelligence. , 2007, Journal of experimental psychology. General.

[18]  Jeffrey N. Rouder,et al.  Modeling Response Times for Two-Choice Decisions , 1998 .

[19]  Andrea Schankin,et al.  Decomposing the relationship between mental speed and mental abilities , 2015 .

[20]  Arndt Bröder,et al.  Empirical validation of the diffusion model for recognition memory and a comparison of parameter-estimation methods , 2015, Psychological research.

[21]  Scott D. Brown,et al.  The hare and the tortoise: emphasizing speed can change the evidence used to make decisions. , 2014, Journal of experimental psychology. Learning, memory, and cognition.

[22]  Andreas Voss,et al.  Social Influence and Perceptual Decision Making , 2014, Personality & social psychology bulletin.

[23]  C. Klein,et al.  Predicting Fluid Intelligence by Components of Reaction Time Distributions from Simple Choice Reaction Time Tasks , 2016 .

[24]  H. Engeland,et al.  Basic Impairments in Regulating the Speed-Accuracy Tradeoff Predict Symptoms of Attention-Deficit/Hyperactivity Disorder , 2010, Biological Psychiatry.

[25]  Roger Ratcliff,et al.  Measuring psychometric functions with the diffusion model. , 2014, Journal of experimental psychology. Human perception and performance.

[26]  E. Wagenmakers,et al.  The Speed-Accuracy Tradeoff in the Elderly Brain: A Structural Model-Based Approach , 2011, The Journal of Neuroscience.

[27]  Andreas Voss,et al.  Experimental validation of the diffusion model based on a slow response time paradigm , 2019, Psychological research.

[28]  R. Ratcliff,et al.  Estimating parameters of the diffusion model: Approaches to dealing with contaminant reaction times and parameter variability , 2002, Psychonomic bulletin & review.

[29]  Thomas V. Wiecki,et al.  The Quality of Response Time Data Inference: A Blinded, Collaborative Assessment of the Validity of Cognitive Models , 2018, Psychonomic Bulletin & Review.

[30]  André Vandierendonck,et al.  A comparison of methods to combine speed and accuracy measures of performance: A rejoinder on the binning procedure , 2016, Behavior Research Methods.

[31]  Han L. J. van der Maas,et al.  On the mean and variance of response times under the diffusion model with an application to parameter estimation , 2009 .

[32]  Markus Janczyk,et al.  Why free choices take longer than forced choices: evidence from response threshold manipulations , 2018, Psychological research.

[33]  Andreas Voss,et al.  A fast numerical algorithm for the estimation of diffusion model parameters , 2008 .

[34]  Roger Ratcliff,et al.  Individual Differences and Fitting Methods for the Two-Choice Diffusion Model of Decision Making. , 2015, Decision.

[35]  Roger Ratcliff,et al.  Individual differences, aging, and IQ in two-choice tasks , 2010, Cognitive Psychology.

[36]  R. Ratcliff,et al.  Evaluating the unequal-variance and dual-process explanations of zROC slopes with response time data and the diffusion model , 2012, Cognitive Psychology.

[37]  R. Ratcliff,et al.  A Diffusion Model Account of Criterion Shifts in the Lexical Decision Task. , 2008, Journal of memory and language.

[38]  Rolf Verleger,et al.  Influence of Time Pressure in a Simple Response Task, a Choice- by-Location Task, and the Simon Task , 2001 .

[39]  M. Brysbaert,et al.  Combining speed and accuracy in cognitive psychology: Is the inverse efficiency score (IES) a better dependent variable than the mean reaction time (RT) and the percentage of errors (PE)? , 2011 .

[40]  A. Voss,et al.  Model Complexity in Diffusion Modeling: Benefits of Making the Model More Parsimonious , 2016, Front. Psychol..

[41]  Francis Tuerlinckx,et al.  Fitting the ratcliff diffusion model to experimental data , 2007, Psychonomic bulletin & review.

[42]  Eric-Jan Wagenmakers,et al.  An EZ-diffusion model for response time and accuracy , 2007, Psychonomic bulletin & review.

[43]  R. Ratcliff,et al.  A diffusion model analysis of the effects of aging on brightness discrimination , 2003, Perception & psychophysics.

[44]  Andreas Voss,et al.  Assessing cognitive processes with diffusion model analyses: a tutorial based on fast-dm-30 , 2015, Front. Psychol..

[45]  E. Wagenmakers,et al.  Diffusion versus linear ballistic accumulation: different models but the same conclusions about psychological processes? , 2010, Psychonomic bulletin & review.

[46]  Risto Näätänen,et al.  Time uncertainty and occurence uncertainty of the stimulus in a simple reaction time task , 1972 .

[47]  R. Ratcliff,et al.  A Diffusion Model Analysis of the Effects of Aging on Recognition Memory Journal of Memory and Language , 2003 .

[48]  E. Wagenmakers,et al.  A diffusion model decomposition of the practice effect , 2009, Psychonomic bulletin & review.

[49]  Scott D. Brown,et al.  The Optimality of Sensory Processing during the Speed–Accuracy Tradeoff , 2012, The Journal of Neuroscience.

[50]  A. Voss,et al.  Interpreting ambiguous stimuli: Separating perceptual and judgmental biases. , 2008 .