Pragmatic Factors in Image Description: The Case of Negations

We provide a qualitative analysis of the descriptions containing negations (no, not, n't, nobody, etc) in the Flickr30K corpus, and a categorization of negation uses. Based on this analysis, we provide a set of requirements that an image description system should have in order to generate negation sentences. As a pilot experiment, we used our categorization to manually annotate sentences containing negations in the Flickr30K corpus, with an agreement score of K=0.67. With this paper, we hope to open up a broader discussion of subjective language in image descriptions.

[1]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[2]  Roger C. Schank,et al.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures , 1978 .

[3]  Xinlei Chen,et al.  Microsoft COCO Captions: Data Collection and Evaluation Server , 2015, ArXiv.

[4]  Laurence R. Horn,et al.  On the semantic properties of logical operators in english' reproduced by the indiana university lin , 1972 .

[5]  Emiel van Miltenburg.  Stereotyping and Bias in the Flickr30K Dataset , 2016, ArXiv.

[6]  Nicu Sebe,et al.  Combining Head Pose and Eye Location Information for Gaze Estimation , 2012, IEEE Transactions on Image Processing.

[7]  Gunnel Tottie,et al.  AFFIXAL AND NON-AFFIXAL NEGATION IN ENGLISH - TWO SYSTEMS IN (ALMOST) COMPLEMENTARY DISTRIBUTION , 2008 .

[8]  Khalil Sima'an,et al.  Multi30K: Multilingual English-German Image Descriptions , 2016, VL@ACL.

[9]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[10]  Antonio Torralba,et al.  Contextual guidance of eye movements and attention in real-world scenes: the role of global features in object search. , 2006, Psychological review.

[11]  Peter Young,et al.  From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions , 2014, TACL.

[12]  C. Fillmore.  FRAME SEMANTICS AND THE NATURE OF LANGUAGE * , 1976 .

[13]  Laurence R. Horn.  A Natural History of Negation , 1989 .

[14]  Nathanael Chambers,et al.  Unsupervised Learning of Narrative Schemas and their Participants , 2009, ACL.

[15]  Roland Hausser,et al.  Principles of Pragmatics , 1989 .

[16]  Camiel J. Beukeboom,et al.  The negation bias: when negations signal stereotypic expectancies. , 2010, Journal of personality and social psychology.

[17]  三嶋 博之.  The theory of affordances , 2008 .

[18]  Roberto Basili,et al.  Automatic induction of FrameNet lexical units , 2008, EMNLP.

[19]  Michael S. Bernstein,et al.  Augur: Mining Human Behaviors from Fiction to Power Interactive Systems , 2016, CHI.

[20]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..