To transform or not to transform: using generalized linear mixed models to analyse reaction time data

Linear mixed-effect models (LMMs) are being increasingly widely used in psychology to analyse multi-level research designs. This feature allows LMMs to address some of the problems identified by Speelman and McGann (2013) about the use of mean data, because they do not average across individual responses. However, recent guidelines for using LMM to analyse skewed reaction time (RT) data collected in many cognitive psychological studies recommend the application of non-linear transformations to satisfy assumptions of normality. Uncritical adoption of this recommendation has important theoretical implications which can yield misleading conclusions. For example, Balota et al. (2013) showed that analyses of raw RT produced additive effects of word frequency and stimulus quality on word identification, which conflicted with the interactive effects observed in analyses of transformed RT. Generalized linear mixed-effect models (GLMM) provide a solution to this problem by satisfying normality assumptions without the need for transformation. This allows differences between individuals to be properly assessed, using the metric most appropriate to the researcher's theoretical context. We outline the major theoretical decisions involved in specifying a GLMM, and illustrate them by reanalysing Balota et al.'s datasets. We then consider the broader benefits of using GLMM to investigate individual differences.

[1]  Donald E. Myers,et al.  Linear and Generalized Linear Mixed Models and Their Applications , 2008, Technometrics.

[2]  J. Nelder,et al.  Generalized Linear Models with Random Effects: Unified Analysis via H-likelihood , 2006 .

[3]  D. Burke,et al.  Why do semantic priming effects increase in old age? A meta-analysis. , 1993, Psychology and aging.

[4]  D. Besner,et al.  Visual word recognition: a multistage activation model. , 1993, Journal of experimental psychology. Learning, memory, and cognition.

[5]  G. Loftus On interpretation of interactions , 1978 .

[6]  Jun Lu,et al.  An introduction to Bayesian hierarchical models with an application in the theory of signal detection , 2005, Psychonomic bulletin & review.

[7]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[8]  Austin F. Frank,et al.  Analyzing linguistic data: a practical introduction to statistics using R , 2010 .

[9]  S. Andrews,et al.  Distinguishing common and task-specific processes in word identification: a matter of some moment? , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[10]  Scott D. Brown,et al.  On the linear relation between the mean and the standard deviation of a response time distribution. , 2007, Psychological review.

[11]  R. Tibshirani,et al.  Generalized Additive Models , 1986 .

[12]  Jiming Jiang Linear and Generalized Linear Mixed Models and Their Applications , 2007 .

[13]  R. Baayen,et al.  Mixed-effects modeling with crossed random effects for subjects and items , 2008 .

[14]  S. Sternberg Memory-scanning: mental processes revealed by reaction-time experiments. , 1969, American scientist.

[15]  David Trafimow,et al.  The mean as a multilevel issue , 2014, Front. Psychol..

[16]  D. Balota,et al.  Moving Beyond the Mean in Studies of Mental Chronometry , 2011 .

[17]  William R. Corson,et al.  Consequences of Failure , 1973 .

[18]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[19]  John J. L. Morton,et al.  Interaction of information in word recognition. , 1969 .

[20]  F. Donders On the speed of mental processes. , 1969, Acta psychologica.

[21]  A. Anastasi Individual differences. , 2020, Annual review of psychology.

[22]  S S Stevens,et al.  On the Theory of Scales of Measurement. , 1946, Science.

[23]  H. H. Clark The language-as-fixed-effect fallacy: A critique of language statistics in psychological research. , 1973 .

[24]  D. Mewhort,et al.  Analysis of Response Time Distributions: An Example Using the Stroop Task , 1991 .

[25]  D. Plaut,et al.  Individual and developmental differences in semantic priming: empirical and computational support for a single-mechanism account of lexical processing. , 2000, Psychological review.

[26]  James A. Bovaird,et al.  On the use of multilevel modeling as an alternative to items analysis in psycholinguistic research , 2007, Behavior research methods.

[27]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[28]  David A Balota,et al.  Additive effects of word frequency and stimulus quality: the influence of trial history and data transformations. , 2013, Journal of experimental psychology. Learning, memory, and cognition.

[29]  Reinhold Kliegl,et al.  Modulation of additive and interactive effects in lexical decision by trial history. , 2013, Journal of experimental psychology. Learning, memory, and cognition.

[30]  William D. Berry,et al.  Testing for Interaction in Binary Logit and Probit Models: Is a Product Term Essential? , 2010 .

[31]  R. H. S. Carpenter,et al.  Neural computation of log likelihood in control of saccadic eye movements , 1995, Nature.

[32]  Kenneth I Forster,et al.  Dynamic adaptation to history of trial difficulty explains the effect of congruency proportion on masked priming. , 2011, Journal of experimental psychology. General.

[33]  L. J. Chapman,et al.  Do Children and the Elderly Show Heightened Semantic Priming? How to Answer the Question. , 1994 .

[34]  Derek Besner,et al.  On the additive effects of stimulus quality and word frequency in lexical decision: evidence for opposing interactive influences revealed by RT distributional analyses. , 2008, Journal of experimental psychology. Learning, memory, and cognition.

[35]  Jeffrey N. Rouder,et al.  A diffusion model account of masking in two-choice letter identification. , 2000, Journal of experimental psychology. Human perception and performance.

[36]  David A. Balota,et al.  Individual differences in the joint effects of semantic priming and word frequency revealed by RT distributional analyses: The role of lexical integrity , 2009 .

[37]  W. Stroup Generalized Linear Mixed Models: Modern Concepts, Methods and Applications , 2012 .

[38]  Roger Ratcliff,et al.  Statistical mimicking of reaction time data: Single-process models, parameter variability, and mixtures , 1995, Psychonomic bulletin & review.

[39]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[40]  Kathleen Carey,et al.  Hospital Length of Stay and Cost: A Multilevel Modeling Analysis , 2002, Health Services and Outcomes Research Methodology.

[41]  S. D. Lima,et al.  General slowing in semantic priming and word recognition. , 1992, Psychology and aging.

[42]  H. Keselman,et al.  Consequences of Assumption Violations Revisited: A Quantitative Review of Alternatives to the One-Way Analysis of Variance F Test , 1996 .

[43]  Francis Tuerlinckx,et al.  A hierarchical approach for fitting curves to response time measurements , 2008, Psychonomic bulletin & review.

[44]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[45]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[46]  R. Ratcliff,et al.  Retrieval Processes in Recognition Memory , 1976 .

[47]  G. Glass,et al.  Consequences of Failure to Meet Assumptions Underlying the Fixed Effects Analyses of Variance and Covariance , 1972 .

[48]  Denis Cousineau,et al.  QMPE: Estimating Lognormal, Wald, and Weibull RT distributions with a parameter-dependent lower bound , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[49]  Scott D. Brown,et al.  The simplest complete model of choice response time: Linear ballistic accumulation , 2008, Cognitive Psychology.

[50]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[51]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .

[52]  W. Schwarz The ex-Wald distribution as a descriptive model of response times , 2001, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[53]  M. Posner Chronometric explorations of mind : the third Paul M. Fitts lectures, delivered at the University of Michigan, September 1976 , 1978 .

[54]  T. Salthouse Speed of behavior and its implications for cog-nition , 1985 .

[55]  M. Masson,et al.  A linear mixed model analysis of masked repetition priming , 2010 .

[56]  Michael R. Harwell,et al.  Summarizing Monte Carlo Results in Methodological Research: The One- and Two-Factor Fixed Effects ANOVA Cases , 1992 .

[57]  D. Bates,et al.  Mixed-Effects Models in S and S-PLUS , 2001 .

[58]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .

[59]  Peter W. Lane,et al.  Generalized linear models in soil science , 2002 .

[60]  David A Balota,et al.  Additive and interactive effects on response time distributions in visual word recognition. , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[61]  D. Balota,et al.  Individual differences in information-processing rate and amount: implications for group differences in response latency. , 1999, Psychological bulletin.

[62]  E. Wagenmakers,et al.  Psychological interpretation of the ex-Gaussian and shifted Wald parameters: A diffusion model analysis , 2009, Psychonomic bulletin & review.

[63]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: part 1.: an account of basic findings , 1988 .

[64]  Marek McGann,et al.  How Mean is the Mean? , 2013, Front. Psychol..

[65]  K. Forster,et al.  More on the language-as-fixed-effect fallacy: Monte Carlo estimates of error rates for F1,F2,F′, and min F′ , 1976 .

[66]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[67]  Robert F. Stanners,et al.  Frequency and visual quality in a word-nonword classification task , 1975 .

[68]  Jeffrey N. Rouder,et al.  Are unshifted distributional models appropriate for response time? , 2005 .

[69]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[70]  F. Carp Handbook of the psychology of aging. , 1977 .

[71]  Charles B. Roosen,et al.  An introduction to multivariate adaptive regression splines , 1995, Statistical methods in medical research.

[72]  L. Hasher,et al.  Automatic and effortful processes in memory. , 1979 .

[73]  G. Mandler,et al.  Retrieval processes in recognition , 1974, Memory & cognition.