Social Cues Modulate Cognitive Status of Discourse Referents

We use visual world eye-tracking to test if a speaker’s eye gaze to a potential antecedent modulates the listener’s interpretation of an ambiguous pronoun. Participants listened to stories that included an ambiguous pronoun, such as “The dolphin kisses the goldfish... He....” During the prepronominal context, an onscreen narrator gazed at one of the two characters. As expected, participants looked more at the subject character overall. However, this was modulated by the narrator’s eye gaze and the amount of time the participant spent looking at the gaze cue. For trials in which participants attended to the narrator’s eye gaze for > 500ms, participants were significantly more likely to interpret the pronoun as referring to the object if the narrator had previously looked at the object. Results suggest that eye gaze – a social cue – can temper even strong linguistic/cognitive biases in pronoun resolution, such as the subject/first-mention bias.

[1]  Agnieszka E. Konopka,et al.  Planning Ahead: How Recent Experience with Structures and Words Changes the Scope of Linguistic Planning. , 2009 .

[2]  P. Hagoort,et al.  Empathy matters: ERP evidence for inter-individual differences in social language processing , 2010, Social cognitive and affective neuroscience.

[3]  A. Kingstone,et al.  The eyes have it! Reflexive orienting is triggered by nonpredictive gaze , 1998 .

[4]  Xiaolin Zhou,et al.  Who is respectful? Effects of social context and individual empathic ability on ambiguity resolution during utterance comprehension , 2015, Front. Psychol..

[5]  Juhani Järvikivi,et al.  Activation and persistence of implicit causality information in spoken language comprehension. , 2010, Experimental psychology.

[6]  Carla L. Hudson Kam,et al.  Knowing ‘who she is’ based on ‘where she is’: The effect of co-speech gesture on pronoun comprehension , 2012, Language and Cognition.

[7]  Juhani Järvikivi,et al.  Ambiguous Pronoun Resolution , 2005, Psychological science.

[8]  Sarah Brown-Schmidt,et al.  The rapid use of gender information: evidence of the time course of pronoun resolution from eyetracking , 2000, Cognition.

[9]  Jeanette K. Gundel,et al.  Cognitive Status and the Form of Referring Expressions in Discourse , 1993 .

[10]  Jennifer E. Arnold,et al.  Effects of psychological attention on pronoun comprehension , 2015, Language, cognition and neuroscience.

[11]  J. V. Berkum,et al.  On the use of verb-based implicit causality in sentence comprehension : Evidence from self-paced reading and eye tracking , 2006 .

[12]  M. Crocker,et al.  Investigating joint attention mechanisms through spoken human–robot interaction , 2011, Cognition.

[13]  Zenzi M. Griffin,et al.  PSYCHOLOGICAL SCIENCE Research Article WHAT THE EYES SAY ABOUT SPEAKING , 2022 .

[14]  Per B. Brockhoff,et al.  lmerTest Package: Tests in Linear Mixed Effects Models , 2017 .

[15]  C. Goodwin Conversational Organization: Interaction Between Speakers and Hearers , 1981 .

[16]  J. Trueswell,et al.  Interpreting pronouns and demonstratives in Finnish: Evidence for a form-specific approach to reference resolution , 2008 .

[17]  Saveria Colonna,et al.  Information structure cues for 4-year-olds and adults: tracking eye movements to visually presented anaphoric referents , 2014 .

[18]  M. Argyle,et al.  Gaze and Mutual Gaze , 1994, British Journal of Psychiatry.

[19]  M. Gernsbacher,et al.  Accessing Sentence Participants: The Advantage of First Mention. , 1988, Journal of memory and language.

[20]  V. Bruce,et al.  Do the eyes have it? Cues to the direction of social attention , 2000, Trends in Cognitive Sciences.

[21]  Jennifer E. Arnold,et al.  The road to understanding is paved with the speaker’s intentions: Cues to the speaker’s attention and intentions affect pronoun comprehension , 2014, Cognitive Psychology.

[22]  S. Brennan,et al.  Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation , 2007 .

[23]  R J Stevenson,et al.  The use of heuristic strategies in the interpretation of pronouns , 1990, Journal of psycholinguistic research.

[24]  Joan Gay Snodgrass,et al.  Naming times for the Snodgrass and Vanderwart pictures , 1996 .

[25]  Barbara J. Grosz,et al.  Pronouns, Names, and the Centering of Attention in Discourse , 1993, Cogn. Sci..

[26]  W. Levelt,et al.  Eye movements during the production of nouns and pronouns , 2001, Memory & cognition.