The Effect of Scene Variation on the Redundant Use of Color in Definite Reference

This study investigates to what extent the amount of variation in a visual scene causes speakers to mention the attribute color in their definite target descriptions, focusing on scenes in which this attribute is not needed for identification of the target. The results of our three experiments show that speakers are more likely to redundantly include a color attribute when the scene variation is high as compared with when this variation is low (even if this leads to overspecified descriptions). We argue that these findings are problematic for existing algorithms that aim to automatically generate psychologically realistic target descriptions, such as the Incremental Algorithm, as these algorithms make use of a fixed preference order per domain and do not take visual scene variation into account.

[1]  A. Meyer,et al.  Tracking the time course of multidimensional stimulus discrimination: Analyses of viewing patterns and processing times during “same”-“different“ decisions , 2002 .

[2]  D R Olson,et al.  Language and thought: aspects of a cognitive theory of semantics. , 1970, Psychological review.

[3]  Chris Mellish,et al.  A Reference Architecture for Natural Language Generation Systems , 2006, Natural Language Engineering.

[4]  M. Tarr,et al.  Becoming a “Greeble” Expert: Exploring Mechanisms for Face Recognition , 1997, Vision Research.

[5]  Alfons Maes,et al.  Overspecification facilitates object identification , 2011 .

[6]  F. Ferreira,et al.  Over-specified referring expressions impair comprehension: An ERP study , 2011, Brain and Cognition.

[7]  Zenzi M. Griffin,et al.  PSYCHOLOGICAL SCIENCE Research Article WHAT THE EYES SAY ABOUT SPEAKING , 2022 .

[8]  M. Tanenhaus,et al.  Introduction to the special issue on language–vision interactions , 2007 .

[9]  Albert Gatt,et al.  Introducing Shared Tasks to NLG: The TUNA Shared Task Evaluation Challenges , 2010, Empirical Methods in Natural Language Generation.

[10]  A. Friedman.  Framing pictures: the role of knowledge in automatized encoding and memory for gist. , 1979, Journal of experimental psychology. General.

[11]  A. Arts,et al.  Overspecification in instructive texts , 2004 .

[12]  Robert Dale,et al.  Building Natural Language Generation Systems: Figures , 2000 .

[13]  Emiel Krahmer,et al.  Toward a Computational Psycholinguistics of Reference Production , 2012, Top. Cogn. Sci..

[14]  A. Friedman.  Framing pictures: the role of knowledge in automatized encoding and memory for gist. , 1979, Journal of experimental psychology. General.

[15]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.

[16]  Marco Baroni,et al.  Predicting Cognitively Salient Modifiers of the Constitutive Parts of Concepts , 2010, CMCL@ACL.

[17]  M. Potter.  Short-term conceptual memory for pictures. , 1976, Journal of experimental psychology. Human learning and memory.

[18]  Robert Dale,et al.  Attribute-Centric Referring Expression Generation , 2010, Empirical Methods in Natural Language Generation.

[19]  T. Pechmann.  Incremental speech production and referential overspecification , 1989 .

[20]  Robert Dale,et al.  Referring Expression Generation through Attribute-Based Heuristics , 2009, ENLG.

[21]  Ielka van der Sluis,et al.  Evaluating algorithms for the Generation of Referring Expressions using a balanced corpus , 2007, ENLG.

[22]  Ehud Reiter,et al.  Book Reviews: Building Natural Language Generation Systems , 2000, CL.

[23]  Robert Dale,et al.  Referring expression generation: what can we learn from human data? , 2009 .

[24]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[25]  Emiel Krahmer,et al.  Factors causing overspecification in definite descriptions , 2011 .

[26]  W. Levelt,et al.  Viewing and naming objects: eye movements during noun phrase production , 1998, Cognition.

[27]  Robert Dale,et al.  Cooking Up Referring Expressions , 1989, ACL.

[28]  Emiel Krahmer,et al.  Introducing shared task evaluation to NLG : The TUNA shared task evaluation challenges , 2010 .

[29]  S. Brennan,et al.  Speakers' eye gaze disambiguates referring expressions early during face-to-face conversation , 2007 .

[30]  Siobhan Chapman.  Logic and Conversation , 2005 .

[31]  Robert Dale,et al.  Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions , 1995, Cogn. Sci..

[32]  Abel G. Oliva,et al.  Gist of a scene , 2005 .

[33]  Ielka van der Sluis,et al.  Generation of Referring Expressions: Assessing the Incremental Algorithm , 2012, Cogn. Sci..

[34]  Emiel Krahmer,et al.  Computational Generation of Referring Expressions: A Survey , 2012, CL.

[35]  Antonio Torralba,et al.  Top-down control of visual attention in object detection , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[36]  Karl G. D. Bailey,et al.  Do speakers and listeners observe the Gricean Maxim of Quantity , 2006 .