Preferential Inspection of Recent Real-World Events Over Future Events: Evidence from Eye Tracking during Spoken Sentence Comprehension

Eye-tracking findings suggest people prefer to ground their spoken language comprehension by focusing on recently seen events more than anticipating future events: When the verb in NP1-VERB-ADV-NP2 sentences was referentially ambiguous between a recently depicted and an equally plausible future clipart action, listeners fixated the target of the recent action more often at the verb than the object that hadn’t yet been acted upon. We examined whether this inspection preference generalizes to real-world events, and whether it is (vs. isn’t) modulated by how often people see recent and future events acted out. In a first eye-tracking study, the experimenter performed an action (e.g., sugaring pancakes), and then a spoken sentence either referred to that action or to an equally plausible future action (e.g., sugaring strawberries). At the verb, people more often inspected the pancakes (the recent target) than the strawberries (the future target), thus replicating the recent-event preference with these real-world actions. Adverb tense, indicating a future versus past event, had no effect on participants’ visual attention. In a second study we increased the frequency of future actions such that participants saw 50/50 future and recent actions. During the verb people mostly inspected the recent action target, but subsequently they began to rely on tense, and anticipated the future target more often for future than past tense adverbs. A corpus study showed that the verbs and adverbs indicating past versus future actions were equally frequent, suggesting long-term frequency biases did not cause the recent-event preference. Thus, (a) recent real-world actions can rapidly influence comprehension (as indexed by eye gaze to objects), and (b) people prefer to first inspect a recent action target (vs. an object that will soon be acted upon), even when past and future actions occur with equal frequency. A simple frequency-of-experience account cannot accommodate these findings.

[1]  G. Humphreys,et al.  The preview search task: Evidence for visual marking , 2006 .

[2]  M. Chun,et al.  Top-Down Attentional Guidance Based on Implicit Learning of Visual Covariation , 1999 .

[3]  Silvia Bernardini,et al.  The WaCky wide web: a collection of very large linguistically processed web-crawled corpora , 2009, Lang. Resour. Evaluation.

[4]  Katherine A. DeLong,et al.  Probabilistic word pre-activation during language comprehension inferred from electrical brain activity , 2005, Nature Neuroscience.

[5]  Craig S. Chapman,et al.  Reaching for the unknown: Multiple target encoding and real-time decision-making in a rapid reach task , 2010, Cognition.

[6]  G. Altmann,et al.  The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye-movements , 2003 .

[7]  Elisabeth Dévière,et al.  Analyzing linguistic data: a practical introduction to statistics using R , 2009 .

[8]  Romi Nijhawan,et al.  Motion extrapolation in catching , 1994, Nature.

[9]  Michael J. Spivey,et al.  Oculomotor mechanisms activated by imagery and memory: eye movements to absent objects , 2001, Psychological research.

[10]  R. H. Baayen,et al.  The CELEX Lexical Database (CD-ROM) , 1996 .

[11]  Christoph Scheepers,et al.  Priming ditransitive structures in comprehension , 2007, Cognitive Psychology.

[12]  Marc Kupietz,et al.  The German Reference Corpus DeReKo: A Primordial Sample for Linguistic Research , 2010, LREC.

[13]  D. Wolpert,et al.  Is the cerebellum a smith predictor? , 1993, Journal of motor behavior.

[14]  Marta Kutas,et al.  The impact of semantic memory organization and sentence context information on spoken language processing by younger and older adults: an ERP study. , 2002, Psychophysiology.

[15]  Christoph Scheepers,et al.  Integration of Syntactic and Semantic Information in Predictive Processing: Cross-Linguistic Evidence from German and English , 2003, Journal of psycholinguistic research.

[16]  Jelena Mirkovic,et al.  Incrementality and Prediction in Human Sentence Processing , 2009, Cogn. Sci..

[17]  John Hale,et al.  The Information Conveyed by Words in Sentences , 2003, Journal of psycholinguistic research.

[18]  Julie C. Sedivy,et al.  Achieving incremental semantic interpretation through contextual representation , 1999, Cognition.

[19]  Colin M. Brown,et al.  Anticipating upcoming words in discourse: evidence from ERPs and reading times. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[20]  Matthew W. Crocker,et al.  The influence of recent scene events on spoken comprehension: Evidence from eye movements , 2007 .

[21]  M. Tanenhaus,et al.  Time Course of Frequency Effects in Spoken-Word Recognition: Evidence from Eye Movements , 2001, Cognitive Psychology.

[22]  J. Saffran Statistical Language Learning , 2003 .

[23]  D. Barr Analyzing ‘visual world’ eyetracking data using multilevel logistic regression , 2008 .

[24]  I. Fischler,et al.  Automatic and attentional processes in the effects of sentence contexts on word recognition , 1979 .

[25]  Alexander Geyken,et al.  dlexDB : eine lexikalische Datenbank für die psychologische und linguistische Forschung , 2011 .

[26]  G. Altmann Language-mediated eye movements in the absence of a visual world: the ‘blank screen paradigm’ , 2004, Cognition.

[27]  M. Grice,et al.  The role of prosody in the interpretation of structural ambiguities: A study of anticipatory eye movements , 2006, Cognition.

[28]  E. Tulving,et al.  STIMULUS INFORMATION AND CONTEXTUAL INFORMATION AS DETERMINANTS OF TACHISTOSCOPIC RECOGNITION OF WORDS. , 1963, Journal of experimental psychology.

[29]  R. Mishra,et al.  Language-mediated prediction is related to reading ability and formal literacy , 2012 .

[30]  Morten H. Christiansen,et al.  Experience and sentence processing: Statistical learning and relative clause comprehension , 2009, Cognitive Psychology.

[31]  J. Trueswell,et al.  The role of discourse context in the processing of a flexible word-order language , 2004, Cognition.

[32]  Matthew W. Crocker,et al.  The influence of the immediate visual context on incremental thematic role-assignment: evidence from eye-movements in depicted events , 2005, Cognition.

[33]  Mirella Lapata,et al.  AMLaP-2005 11th Annual Conference on Architectures and Mechanisms for Language Processing , 2005 .

[34]  Kara D. Federmeier Thinking ahead: the role and roots of prediction in language comprehension. , 2007, Psychophysiology.

[35]  Gary E. Raney,et al.  Eye movement control in reading and visual search: Effects of word frequency , 1996, Psychonomic bulletin & review.

[36]  M. Tanenhaus,et al.  Actions and affordances in syntactic ambiguity resolution. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[37]  Peter Wittenburg,et al.  Annotation by Category: ELAN and ISO DCR , 2008, LREC.

[38]  Robert J Hartsuiker,et al.  Looking, language, and memory: bridging research from the visual world and visual search paradigms. , 2011, Acta psychologica.

[39]  T D STERLING,et al.  Sources of contextual constraint upon words in sentences. , 1959, Journal of experimental psychology.

[40]  Christoph Scheepers,et al.  Syntactic priming in comprehension: the role of argument order and animacy. , 2008, Journal of experimental psychology. Learning, memory, and cognition.

[41]  C. Urgesi,et al.  Action anticipation and motor resonance in elite basketball players , 2008, Nature Neuroscience.

[42]  Austin F. Frank,et al.  Analyzing linguistic data: a practical introduction to statistics using R , 2010 .

[43]  R N Aslin,et al.  Statistical Learning by 8-Month-Old Infants , 1996, Science.

[44]  Jeffrey M. Zacks,et al.  Event perception: a mind-brain perspective. , 2007, Psychological bulletin.

[45]  R. Levy Expectation-based syntactic comprehension , 2008, Cognition.

[46]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[47]  Michael P. Kaschak,et al.  Recent experience affects the strength of structural priming , 2006, Cognition.

[48]  Michael I. Jordan,et al.  An internal model for sensorimotor integration. , 1995, Science.

[49]  Michael J. Berry,et al.  Anticipation of moving stimuli by the retina , 1999, Nature.

[50]  G. Altmann,et al.  Incremental interpretation at verbs: restricting the domain of subsequent reference , 1999, Cognition.

[51]  Philipp Koehn,et al.  Europarl: A Parallel Corpus for Statistical Machine Translation , 2005, MTSUMMIT.

[52]  Christoph Scheepers,et al.  Syntactic priming of relative clause attachments: persistence of structural configuration in sentence production , 2003, Cognition.

[53]  M. Pickering,et al.  Do people use language production to make predictions during comprehension? , 2007, Trends in Cognitive Sciences.

[54]  M. MacDonald,et al.  Experience and grammatical agreement: Statistical learning shapes number agreement production , 2010, Cognition.