The cave of Shadows. Addressing the human factor with generalized additive mixed models

Abstract Generalized additive mixed models are introduced as an extension of the generalized linear mixed model which makes it possible to deal with temporal autocorrelational structure in experimental data. This autocorrelational structure is likely to be a consequence of learning, fatigue, or the ebb and flow of attention within an experiment (the ‘human factor’). Unlike molecules or plots of barley, subjects in psycholinguistic experiments are intelligent beings that depend for their survival on constant adaptation to their environment, including the environment of an experiment. Three data sets illustrate that the human factor may interact with predictors of interest, both factorial and metric. We also show that, especially within the framework of the generalized additive model, in the nonlinear world, fitting maximally complex models that take every possible contingency into account is ill-advised as a modeling strategy. Alternative modeling strategies are discussed for both confirmatory and exploratory data analysis.

[1]  G. Box.  Science and Statistics , 1976 .

[2]  R. Tibshirani,et al.  Generalized additive models for medical research , 1986, Statistical methods in medical research.

[3]  Xiaolin Zhou,et al.  Center of mass attracts attention , 2006, Neuroreport.

[4]  R. Harald Baayen,et al.  Morphological effects in auditory word recognition: Evidence from Danish , 2008 .

[5]  Jeruen E. Dery,et al.  On Experience-Driven Semantic Judgments: A Case Study on the Oneiric Reference Constraint , 2015 .

[6]  R. Tibshirani,et al.  Generalized Additive Models , 1986 .

[7]  G. Francis.  Publication bias in "Red, rank, and romance in women viewing men," by Elliot et al. (2010). , 2013, Journal of experimental psychology. General.

[8]  B. Rossion,et al.  Fixation Patterns During Recognition of Personally Familiar and Unfamiliar Faces , 2010, Front. Psychology.

[9]  K I Forster,et al.  The potential for experimenter bias effects in word recognition experiments , 2000, Memory & cognition.

[10]  R. Baayen,et al.  Analyzing Reaction Times , 2010 .

[11]  Dave F. Kleinschmidt,et al.  Robust speech perception: recognize the familiar, generalize to the similar, and adapt to the novel. , 2015, Psychological review.

[12]  Reinhold Kliegl,et al.  Experimental Effects and Individual Differences in Linear Mixed Models: Estimating the Relationship between Spatial, Object, and Attraction Effects in Visual Attention , 2010, Front. Psychology.

[13]  Simon N. Wood,et al.  A simple test for random effects in regression models , 2013 .

[14]  H. Baayen,et al.  Vietnamese compounds show an anti-frequency effect in visual lexical decision , 2015 .

[15]  G. Box.  Robustness in the Strategy of Scientific Model Building. , 1979 .

[16]  S. Wood.  On p-values for smooth components of an extended generalized additive model , 2013 .

[17]  Jon Driver,et al.  Shifting visual attention between objects and locations: Normality and pathology. , 1994 .

[18]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[19]  S. Segalowitz,et al.  RT coefficient of variation is differentially sensitive to executive control involvement in an attention switching task , 1999 .

[20]  D. Bates,et al.  Mixed-Effects Models in S and S-PLUS , 2001 .

[21]  Susanne Gahl,et al.  The Ecclesiastes principle in language change , 2017 .

[22]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[23]  X. Lin,et al.  Inference in generalized additive mixed modelsby using smoothing splines , 1999 .

[24]  Anne Cutler,et al.  The Quarterly Journal of Experimental Psychology Baldey: a Database of Auditory Lexical Decisions , 2022 .

[25]  A. Welford.  Choice reaction time: Basic concepts , 1980 .

[26]  Tomasz Burzykowski,et al.  Linear Mixed-Effects Models Using R: A Step-by-Step Approach , 2013 .

[27]  R. Baayen,et al.  Regular morphologically complex neologisms leave detectable traces in the mental lexicon , 2007 .

[28]  Lee H. Wurm,et al.  What residualizing predictors in regression analyses does (and what it does not do) , 2014 .

[29]  W. Cleveland.  Robust Locally Weighted Regression and Smoothing Scatterplots , 1979 .

[30]  F. Tuerlinckx,et al.  Linear mixed modelling for data from a double mixed factorial design with covariates: a case‐study on semantic categorization response times , 2014 .

[31]  S. Wood.  Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models , 2011 .

[32]  Trevor Hastie,et al.  An Introduction to Statistical Learning , 2013, Springer Texts in Statistics.

[33]  Sidney J. Segalowitz,et al.  Skilled performance, practice, and the differentiation of speed-up from automatization effects: Evidence from second language word recognition , 1993, Applied Psycholinguistics.

[34]  Simon N Wood,et al.  Just Another Gibbs Additive Modeler: Interfacing JAGS and mgcv , 2016, 1602.02539.

[35]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[36]  Melody Dye,et al.  The Effects of Feature-Label-Order and Their Implications for Symbolic Learning , 2010, Cogn. Sci..

[37]  G. Tutz,et al.  An introduction to recursive partitioning: rationale, application, and characteristics of classification and regression trees, bagging, and random forests. , 2009, Psychological methods.

[38]  Michael C. Frank,et al.  Estimating the reproducibility of psychological science , 2015, Science.

[39]  James E. Helmreich.  Regression Modeling Strategies with Applications to Linear Models, Logistic and Ordinal Regression and Survival Analysis (2nd Edition) , 2016 .

[40]  Jacob Cohen.  The Cost of Dichotomization , 1983 .

[41]  G. Box.  An analysis of transformations (with discussion) , 1964 .

[42]  R. Baayen,et al.  Lexical statistics and lexical processing: semantic density, information complexity, sex, and irregularity in Dutch , 2005 .

[43]  Reinhold Kliegl,et al.  Smoothing Spline ANOVA Decomposition of Arbitrary Splines: An Application to Eye Movements in Reading , 2015, PloS one.

[44]  D. Kimura.  Sex and cognition , 1999 .

[45]  Laura Winther Balling,et al.  Probability and surprisal in auditory comprehension of morphologically complex words , 2012, Cognition.

[46]  Jacolien van Rij,et al.  Autocorrelated Errors in Experimental Data in the Language Sciences: Some Solutions Offered by Generalized Additive Mixed Models , 2016, 1601.02043.

[47]  D. A. Kenny,et al.  Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli. , 2014, Journal of experimental psychology. General.

[48]  D. Bates,et al.  Parsimonious Mixed Models , 2015, 1506.04967.

[49]  R. Baayen,et al.  Learning is not decline: The mental lexicon as a window into cognition across the lifespan , 2014 .

[50]  Donald E. Broadbent,et al.  Decision and stress , 1971 .

[51]  Koch Sigmund Ed,et al.  Psychology: A Study of A Science , 1962 .

[52]  D. Bates,et al.  Balancing Type I Error and Power in Linear Mixed Models , 2015, 1511.01864.

[53]  P. Sander,et al.  Rogue Males: Sex Differences in Psychology Students , 2017 .

[54]  R. Baayen,et al.  Effects of primary and secondary morphological family size in monolingual and bilingual word processing , 2014 .

[55]  Michael Ramscar,et al.  Granularity and the acquisition of grammatical gender: How order-of-acquisition affects what gets learned , 2012, Cognition.

[56]  Kristopher J Preacher,et al.  On the practice of dichotomization of quantitative variables. , 2002, Psychological methods.

[57]  R. Rafal,et al.  Shifting visual attention between objects and locations: evidence from normal and parietal lesion subjects. , 1994, Journal of experimental psychology. General.

[58]  Reinhold Kliegl,et al.  Modulation of additive and interactive effects in lexical decision by trial history. , 2013, Journal of experimental psychology. Learning, memory, and cognition.

[59]  Subhash R. Lele,et al.  Estimability and Likelihood Inference for Generalized Linear Mixed Models Using Data Cloning , 2010 .

[60]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[61]  D. Cox,et al.  An Analysis of Transformations , 1964 .

[62]  K. Forster,et al.  More on the language-as-fixed-effect fallacy: Monte Carlo estimates of error rates for F1,F2,F′, and min F′ , 1976 .

[63]  D. Barr,et al.  Random effects structure for confirmatory hypothesis testing: Keep it maximal. , 2013, Journal of memory and language.

[64]  D. Gilden.  Cognitive emissions of 1/f noise. , 2001, Psychological review.

[65]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[66]  Andries F. Sanders,et al.  Elements of Human Performance: Reaction Processes and Attention in Human Skill , 1998 .

[67]  Robbin A. Miranda,et al.  Sex Differences in the Neurocognition of Language , 2007 .

[68]  C. J. Marsolek.  What antipriming reveals about priming , 2008, Trends in Cognitive Sciences.

[69]  Douglas Nychka,et al.  Bayesian Confidence Intervals for Smoothing Splines , 1988 .

[70]  Paul J. Rathouz,et al.  Semiparametric generalized linear models , 2008 .

[71]  D L Gilden,et al.  1/f noise in human cognition. , 1995, Science.

[72]  S. Wood,et al.  Coverage Properties of Confidence Intervals for Generalized Additive Model Components , 2012 .

[73]  G. Francis.  Publication bias and the failure of replication in experimental psychology , 2012, Psychonomic bulletin & review.

[74]  Jacolien van Rij,et al.  itsadug: Interpreting Time Series and Autocorrelated Data Using GAMMs , 2015 .

[75]  Robert Schreuder,et al.  Producing inflected verbs: A picture naming study , 2010 .

[76]  J. Hulstijn,et al.  Automaticity in second language learning , 2005 .

[77]  S. Lupker,et al.  Sequential effects in naming: a time-criterion account. , 2001, Journal of experimental psychology. Learning, memory, and cognition.

[78]  S. Wood,et al.  Generalized additive models for large data sets , 2015 .

[79]  R. Baayen.  A real experiment is a factorial experiment , 2010 .

[80]  R. Baayen,et al.  Frequency in lexical processing , 2016 .