Modelling individual response time effects between and within experimental speed conditions: A GLMM approach for speeded tests

Completing test items under multiple speed conditions avoids the performance measure being confounded with individual differences in the speed-accuracy compromise, and offers insights into the response process, that is, how response time relates to the probability of a correct response. This relation is traditionally represented by two conceptually different functions: the speed-accuracy trade-off function (SATF) across conditions relating the condition average response time to the condition average of accuracy, and the conditional accuracy function (CAF) within a condition describing accuracy conditional on response time. Using a generalized linear mixed modelling approach, we propose an item response modelling framework that is suitable for item response and response time data from experimental speed conditions. The proposed SATF and CAF model accommodates response time effects between conditions (i.e., person and item SATF slope) and within conditions (i.e., residual CAF slopes), captures person and item differences in these effects, and is suitable for measures with a strong speed component. Moreover, for a single condition a CAF model is proposed distinguishing person, item and residual CAF. The properties of the models are illustrated with an empirical example.

[1]  G. Maris,et al.  Speed-Accuracy Response Models: Scoring Rules based on Response Time and Accuracy , 2012 .

[2]  Abe D. Hofman,et al.  The estimation of item response models with the lmer function from the lme4 package in R , 2011 .

[3]  Douglas M. Bates,et al.  Estimating the Multilevel Rasch Model: With the lme4 Package , 2007 .

[4]  M. Davison,et al.  Modeling Individual Differences in Numerical Reasoning Speed as a Random Effect of Response Time Limits , 2011 .

[5]  Jeffrey J. Walczyk.  The Interplay Between Automatic and Control Processes in Reading , 2000 .

[6]  Wayne A. Wickelgren,et al.  Speed-accuracy tradeoff and information processing dynamics , 1977 .

[7]  Charles A. Perfetti,et al.  Reading Ability: Lexical Quality to Comprehension , 2007 .

[8]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .

[9]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[10]  Frank Goldhammer,et al.  Measuring Ability, Speed, or Both? Challenges, Psychometric Solutions, and What Can Be Gained From Experimental Control , 2015, Measurement : interdisciplinary research and perspectives.

[11]  M. Jeon.  Differential Response Speed: Is it Really a Nuisance? , 2015 .

[12]  Ian Dennis,et al.  The speed–error trade‐off problem in psychometric testing , 1996 .

[13]  Heiko Rölke,et al.  The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment. , 2014 .

[14]  Tobias Richter,et al.  Prozessbezogene Diagnostik von Lesefähigkeiten bei Grundschulkindern , 2012 .

[15]  Mollie E. Brooks,et al.  Generalized linear mixed models: a practical guide for ecology and evolution. , 2009, Trends in ecology & evolution.

[16]  Timothy A. Salthouse,et al.  Interpreting Reaction Time Measures in Between-Group Comparisons , 2002, Journal of clinical and experimental neuropsychology.

[17]  W. D. Linden,et al.  Conceptual Issues in Response-Time Modeling. , 2009 .

[18]  R. H. Klein Entink,et al.  A Multivariate Multilevel Approach to the Modeling of Accuracy and Speed of Test Takers , 2008, Psychometrika.

[19]  E. Hunt.  Mechanics of Verbal Ability. , 1978 .

[20]  M. Bolsinova,et al.  Can Response Speed Be Fixed Experimentally, and Does This Lead to Unconfounded Measurement of Ability? , 2015 .

[21]  Eric J. Johnson,et al.  Adaptive Strategy Selection in Decision Making. , 1988 .

[22]  Benjamin M. Bolker,et al.  Ecological Models and Data in R , 2008 .

[23]  P. Boeck,et al.  Modelling Conditional Dependence Between Response Time and Accuracy , 2017, Psychometrika.

[24]  T. Salthouse,et al.  Adult age and the speed-accuracy trade-off. , 1979, Ergonomics.

[25]  Jochen Ranger,et al.  The case of dependency of responses and response times: A modeling approach based on standard latent trait models , 2012 .

[26]  Johannes Naumann,et al.  Time-on-task effects in digital reading are non-linear and moderated by persons' skills and tasks' demands , 2017 .

[27]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[28]  Edward E. Roskam,et al.  Models for Speed and Time-Limit Tests , 1997 .

[29]  G. Molenberghs,et al.  Likelihood Ratio, Score, and Wald Tests in a Constrained Parameter Space , 2007 .

[30]  R. Ratcliff.  Modeling response signal and response time data , 2006, Cognitive Psychology.

[31]  J WIM,et al.  A HIERARCHICAL FRAMEWORK FOR MODELING SPEED AND ACCURACY ON TEST ITEMS , 2007 .

[32]  David F. Lohman,et al.  Individual differences in errors and latencies on cognitive tasks , 1989 .

[33]  Geert Molenberghs,et al.  The gradient function as an exploratory goodness-of-fit assessment of the random-effects distribution in mixed models. , 2013, Biostatistics.

[34]  Daniel Callison,et al.  Time on Task. , 1998 .

[35]  G. Smith,et al.  Slowness and age: speed-accuracy mechanisms. , 1995, Psychology and aging.

[36]  Frank Goldhammer,et al.  Controlling Individuals’ Time Spent on Task in Speeded Performance Measures , 2014 .

[37]  Zhiying Pan,et al.  Goodness‐of‐Fit Methods for Generalized Linear Mixed Models , 2005, Biometrics.

[38]  Cornelis A.W. Glas,et al.  Statistical Tests of Conditional Independence Between Responses and/or Response Times on Test Items , 2010 .

[39]  Richard P. Heitz,et al.  The speed-accuracy tradeoff: history, physiology, methodology, and behavior , 2014, Front. Neurosci..

[40]  D. Molenaar.  The Value of Response Times in Item Response Modeling , 2015 .

[41]  F. Vaida,et al.  Conditional Akaike information for mixed-effects models , 2005 .

[42]  O. Wilhelm,et al.  The relation of speeded and unspeeded reasoning with mental speed , 2002 .

[43]  S. Greven,et al.  On the behaviour of marginal and conditional AIC in linear mixed models , 2010 .

[44]  Gerard J. P. Van Breukelen,et al.  Psychometric Modeling of response speed and accuracy with mixed and conditional regression , 2005 .