Image Retrieval by Emotional Semantics: A Study of Emotional Space and Feature Extraction

Image emotional semantic research is a promising and challenging issue. This paper analyzes the emotional space and builds a novel scheme that can annotate the image emotion semantic automatically and realize emotional image retrieval. Based on psychological experiments evaluating evoked feelings for art paintings, we first identify an orthogonal three-dimension emotional factor space of images through 12 pairs of emotional words. Then, the following three novel image features are designed for each emotional factor to predict it. They are luminance-warm-cool fuzzy histogram, saturation-warm-cool fuzzy histogram integrated with color contrast and luminance contrast integrated with edge sharpness. The values of emotional factors can be predicted from the image features automatically by using support vector machine of regression. Finally, we design and implement an emotion-based image retrieval system, which enables the users to perform retrieval using emotional semantic words. Experimental results show the effectiveness of our model.

[1]  C. Spearman.  General intelligence Objectively Determined and Measured , 1904 .

[2]  J. M. Kittross.  The measurement of meaning , 1959 .

[3]  Masafumi Hagiwara,et al.  Image query by impression words-the IQI system , 1998 .

[4]  Toshikazu Kato,et al.  Image retrieval system using impression words , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[5]  Alberto Del Bimbo,et al.  Semantics in Visual Information Retrieval , 1999, IEEE Multim..

[6]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Nadia Bianchi-Berthouze,et al.  K-DIME: An Affective Image Filtering System , 2003, IEEE Multim..

[8]  James Ze Wang,et al.  Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[10]  Sethuraman Panchanathan,et al.  Indexing natural images for retrieval based on Kansei factors , 2004, IS&T/SPIE Electronic Imaging.

[11]  L. Ou,et al.  A study of colour emotion and colour preference. Part I: Colour emotions for single colours , 2004 .

[12]  Zhang Jian-chao,et al.  Image emotional classification: static vs. dynamic , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[13]  Aleksandra Mojsilovic,et al.  Semantic-Friendly Indexing and Quering of Images Based on the Extraction of the Objective Semantic Cues , 2004, International Journal of Computer Vision.

[14]  Alan Hanjalic,et al.  Affective video content representation and modeling , 2005, IEEE Transactions on Multimedia.

[15]  Pankoo Kim,et al.  Matching Colors with KANSEI Vocabulary Using Similarity Measure Based on WordNet , 2005, ICCSA.

[16]  Wei-Ning Wang,et al.  Image emotional semantic query based on color semantic description , 2005, 2005 International Conference on Machine Learning and Cybernetics.