An Image Impression Estimation System using Adjectives

This paper addresses the challenge of creating a new system to estimate the impression of an image. The proposed system combines the human annotated tags of images and an image classification method to discover “showing a photo, what are people looking at?”. Then, to tackle the challenge “what are they thinking about the one they look at?”, the semantic association strengths between adjectives and image keywords are computed by pointwise mutual information (PMI) and the pattern frequencies using a machine learning approach. To select the output, we use a rank aggregation method, Borda’s method, to generate an acceptable ranking for a given set of rankings and the top na adjectives (in this paper na is 5) are chosen according to the estimated values. The main contribution of this method is to design an effective method for estimating the association of the impression adjectives with images. We evaluated the proposed approach using two tasks: the first one is the performance of the task of keyword extraction and the second one is the efficiency of the proposed method. For the performance of the proposed method, we carried out subjective experiments and obtained fairly good results.

[1]  O. Pos,et al.  Facial Expressions, Colours and Basic Emotions , 2012 .

[2]  Miki Haseyama,et al.  A cross-modal approach for extracting semantic relationships of concepts from an image database , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  Fred Popowich,et al.  Opinion Polarity Identification through Adjectives , 2010, ArXiv.

[4]  Yi Yang,et al.  Augmenting Image Descriptions Using Structured Prediction Output , 2014, IEEE Transactions on Multimedia.

[5]  Liming Chen,et al.  Semantic Bag-of-Words Models for Visual Concept Detection and Annotation , 2012, 2012 Eighth International Conference on Signal Image Technology and Internet Based Systems.

[6]  Shigenobu Kobayashi,et al.  The aim and method of the color image scale , 2009 .

[7]  V. Udhayakumar,et al.  A Web Search Engine-Based Approach to Measure Semantic Similarity between Words , 2015 .

[8]  Michael Gasser,et al.  Learning Nouns and Adjectives: A Connectionist Account , 1998 .

[9]  Michael R. Lyu,et al.  Bridging the Semantic Gap Between Image Contents and Tags , 2010, IEEE Transactions on Multimedia.

[10]  H. P. Young,et al.  An axiomatization of Borda's rule , 1974 .

[11]  Moni Naor,et al.  Rank aggregation methods for the Web , 2001, WWW '01.

[12]  Mark C. Baker Lexical Categories: Verbs, Nouns and Adjectives , 2003 .

[13]  Chris Brew The Cambridge Grammar of the English Language , 2003 .

[14]  Danushka Bollegala,et al.  A Web Search Engine-Based Approach to Measure Semantic Similarity between Words , 2011, IEEE Transactions on Knowledge and Data Engineering.

[15]  J. Xin,et al.  Cross-regional comparison of colour emotions Part II: Qualitative analysis , 2004 .

[16]  Diego Reforgiato Recupero,et al.  AVA: Adjective-Verb-Adverb Combinations for Sentiment Analysis , 2008, IEEE Intelligent Systems.

[17]  Yejin Choi,et al.  Baby talk: Understanding and generating simple image descriptions , 2011, CVPR 2011.

[18]  Ahmet Aker,et al.  Generating Image Descriptions Using Dependency Relational Patterns , 2010, ACL.

[19]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[20]  Mirella Lapata,et al.  Verb Class Disambiguation Using Informative Priors , 2004, CL.