The use of visual context during the production of referring expressions

Most theories of reference assume that a referent's saliency in the linguistic context determines the choice of referring expression. However, it is less clear whether cognitive factors relating to the nonlinguistic context also have an effect. We investigated whether visual context influences the choice of a pronoun over a repeated noun phrase when speakers refer back to a referent in a preceding sentence. In Experiment 1, linguistic mention as well as visual presence of a competitor with the same gender as the referent resulted in fewer pronouns for the referent, suggesting that both linguistic and visual context determined the choice of referring expression. Experiment 2 showed that even when the competitor had a different gender from the referent, its visual presence reduced pronoun use, indicating that visual context plays a role even if the use of a pronoun is unambiguous. Thus, both linguistic and nonlinguistic information affect the choice of referring expression.

[1]  P. Johnson-Laird,et al.  Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness , 1985 .

[2]  Julie C. Sedivy,et al.  Evidence of Perspective-Taking Constraints in Children's On-Line Reference Resolution , 2002, Psychological science.

[3]  D R Olson,et al.  Language and thought: aspects of a cognitive theory of semantics. , 1970, Psychological review.

[4]  M. Tanenhaus,et al.  Watching the eyes when talking about size: An investigation of message formulation and utterance planning , 2006 .

[5]  T. Givon.  Topic Continuity in Discourse , 1983 .

[6]  Victor S. Ferreira,et al.  How do speakers avoid ambiguous linguistic expressions? , 2005, Cognition.

[7]  Roger P. G. van Gompel,et al.  Choosing anaphoric expressions : do people take into account likelihood of reference? , 2010 .

[8]  L. Gleitman,et al.  On the give and take between event apprehension and utterance formulation. , 2007, Journal of memory and language.

[9]  Russell S. Tomlin,et al.  Language and conceptualization: Mapping conceptual representations into linguistic representations: the role of attention in grammar , 1997 .

[10]  C. Fletcher.  Markedness and topic continuity in discourse processing , 1984 .

[11]  Jeanette K. Gundel,et al.  Cognitive Status and the Form of Referring Expressions in Discourse , 1993 .

[12]  A Pollatsek,et al.  On the use of counterbalanced designs in cognitive research: a suggestion for a better and more powerful analysis. , 1995, Journal of experimental psychology. Learning, memory, and cognition.

[13]  P. Gordon,et al.  Pronouns, Passives, and Discourse Coherence , 1995 .

[14]  Carl Pollard,et al.  A Centering Approach to Pronouns , 1987, ACL.

[15]  Mira Ariel.  Accessing Noun-Phrase Antecedents , 1990 .

[16]  Barbara J. Grosz,et al.  Pronouns, Names, and the Centering of Attention in Discourse , 1993, Cogn. Sci..

[17]  A. Almor,et al.  Noun-phrase anaphors and focus: the informational load hypothesis. , 1999, Psychological review.

[18]  W. Chafe.  Givenness, contrastiveness, definiteness, subjects, topics, and point of view , 1976 .

[19]  B. Keysar,et al.  When do speakers take into account common ground? , 1996, Cognition.

[20]  Sarah Brown-Schmidt,et al.  The rapid use of gender information: evidence of the time course of pronoun resolution from eyetracking , 2000, Cognition.

[21]  Robert L. Mason,et al.  Statistical Principles in Experimental Design , 2003 .

[22]  Kenneth I Forster,et al.  DMDX: A Windows display program with millisecond accuracy , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[23]  Rosemary J. Stevenson,et al.  Thematic roles, focus and the representation of events , 1994 .

[24]  Jennifer E. Arnold,et al.  The Effect of Thematic Roles on Pronoun Use and Frequency of Reference Continuation , 2001 .

[25]  Julie C. Sedivy,et al.  Pragmatic Versus Form-Based Accounts of Referential Contrast: Evidence for Effects of Informativity Expectations , 2003, Journal of psycholinguistic research.

[26]  Zenzi M. Griffin,et al.  PSYCHOLOGICAL SCIENCE Research Article WHAT THE EYES SAY ABOUT SPEAKING , 2022 .

[27]  Scott Weinstein,et al.  Centering: A Framework for Modeling the Local Coherence of Discourse , 1995, CL.

[28]  Siobhan Chapman.  Logic and Conversation , 2005 .

[29]  D. E. Irwin,et al.  Minding the clock , 2003 .

[30]  Anne H. Anderson,et al.  The Accessibility of Pronominal Antecedents as a Function of Episode Shifts in Narrative Text , 1983 .

[31]  H. H. Clark,et al.  Conceptual pacts and lexical choice in conversation. , 1996, Journal of experimental psychology. Learning, memory, and cognition.

[32]  Wietske Vonk,et al.  The use of referential expressions in structuring discourse , 1992 .

[33]  Ron Zacharski,et al.  Pronouns without NP antecedents: how do we know when a pronoun is referential? , 2005 .

[34]  S. Brennan.  Centering Attention in Discourse. , 1995 .

[35]  Jennifer E. Arnold,et al.  The effect of additional characters on choice of referring expression: Everyone counts. , 2007, Journal of memory and language.

[36]  T. Givón.  Topic continuity in discourse: An introduction , 1983 .

[37]  Robbert-Jan Beun,et al.  Object reference in a shared domain of conversation , 1998 .

[38]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[39]  Randall Hendrick,et al.  The Representation and Processing of Coreference in Discourse , 1998, Cogn. Sci..

[40]  W. Chafe.  The Pear Stories: Cognitive, Cultural and Linguistic Aspects of Narrative Production , 1980 .