Out of the Cage of Shadows

Unlike molecules or plots of barley, subjects in psycholinguistic experiments are intelligent beings that depend for their survival on constant adaptation to their environment. This study presents three data sets documenting the presence of adaptive processes in psychological experiments. These adaptive processes leave a statistical footprint in the form of autocorrelations in the residual error associated with by-subject time series of trial-to-trial responses. Generalized additive mixed models (GAMMs) provide a unified framework within which both factorial predictors and covariates given with the experimental design, as well as non-linear random effects and interactions with experimental time can be uncovered and evaluated. GAMMs not only provide substantially improved fits to experimental data with time series structure, but also provide improved insight into predictors of theoretical interest, as well as a more refined window on the random effects structure. Our results challenge the standard advocated by Barr et al. (2013). The analytical cage of the maximal linear mixed model to which this standard confines the analyst is motivated by simulation studies which presuppose experimental data to be sterile, and free of any adaptive processes. However, when adaptive processes are present in real data, the simulation results of Barr et al. are no longer informative. For such data, the method of analysis cannot be purely design-driven, but must be in part driven by the data.