Comparing the time-course of processing initially ambiguous and unambiguous German SVO/OVS sentences in depicted events

Publisher Summary The monitoring of eye movements in scenes containing objects has revealed that the type of visual referential context rapidly influences the initial structuring and interpretation of a concurrently presented utterance. Studies by Knoeferle, Crocker, Scheepers, and Pickering have extended this work by examining the influence of richer visual environments that contained explicitly depicted agent–action–patient events in addition to characters and objects. The Coordinated Interplay Account by Knoeferle and Crocker predicts that the time course with which a depicted event influences thematic role assignment depends on when that depicted event is identified as relevant by the utterance. Eye movements were monitored in a scene during the comprehension of German utterances that differed with respect to when they identified a relevant depicted event for thematic role assignment and structuring of the utterance. Findings confirmed the coordinated interplay account: Gaze patterns revealed differences in the time course with which a depicted event triggered thematic role assignment depending on whether that event was identified as relevant for comprehension early or late by the utterance.

[1]  Shweta Narayan,et al.  Simulated Action in an Embodied Construction Grammar , 2004 .

[2]  R. H. Baayen,et al.  The CELEX Lexical Database (CD-ROM) , 1996 .

[3]  Deb Roy,et al.  Towards situated speech understanding: visual context priming of language models , 2005, Comput. Speech Lang..

[4]  Matthew W. Crocker,et al.  The influence of the immediate visual context on incremental thematic role-assignment: evidence from eye-movements in depicted events , 2005, Cognition.

[5]  M. Crocker Computational Psycholinguistics: An Interdisciplinary Approach to the Study of Language , 1995 .

[6]  G. Altmann,et al.  The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye-movements , 2003 .

[7]  P. Knoeferle,et al.  The role of visual scenes in spoken language comprehension : evidence from eye-tracking , 2005 .

[8]  Roger M. Cooper,et al.  The control of eye fixation by the meaning of spoken language: A new methodology for the real-time investigation of speech perception, memory, and language processing. , 1974 .

[9]  Thomas G. Bever,et al.  Sentence Comprehension: The Integration of Habits and Rules , 2001 .

[10]  Tim Shallice,et al.  Soar and the case for unified theories of cognition , 1995, Cognition.

[11]  Fernanda Ferreira,et al.  Scene Perception for Psycholinguists. , 2004 .

[12]  G. Altmann,et al.  Incremental interpretation at verbs: restricting the domain of subsequent reference , 1999, Cognition.

[13]  E. Marshall,et al.  NIMH: caught in the line of fire without a general , 1995, Science.

[14]  Thomas F Münte,et al.  Cerebral Cortex Advance Access published July 21, 2007 Visual Scenes Trigger Immediate Syntactic Reanalysis: Evidence from ERPs during Situated Spoken Comprehension , 2022 .

[15]  Lawrence W. Barsalou,et al.  Language comprehension: Archival memory or preparation for situated action? , 1999 .

[16]  Christoph Scheepers,et al.  Integration of Syntactic and Semantic Information in Predictive Processing: Cross-Linguistic Evidence from German and English , 2003, Journal of psycholinguistic research.

[17]  M. Tanenhaus,et al.  Actions and affordances in syntactic ambiguity resolution. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[18]  D. C. Howell Statistical Methods for Psychology , 1987 .

[19]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[20]  Julie C. Sedivy,et al.  Achieving incremental semantic interpretation through contextual representation , 1999, Cognition.

[21]  Julie C. Sedivy,et al.  Eye movements and spoken language comprehension: Effects of visual context on syntactic ambiguity resolution , 2002, Cognitive Psychology.

[22]  Mark Steedman,et al.  Interaction with context during human sentence processing , 1988, Cognition.

[23]  R. Jackendoff Foundations of Language: Brain, Meaning, Grammar, Evolution , 2002 .

[24]  Matthew W. Crocker,et al.  The Coordinated Interplay of Scene, Utterance, and World Knowledge: Evidence From Eye Tracking , 2006, Cogn. Sci..

[25]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[26]  A. Newell Unified Theories of Cognition , 1990 .

[27]  Maryellen C. MacDonald,et al.  The lexical nature of syntactic ambiguity resolution , 1994 .