Classification of affective semantics in images based on discrete and dimensional models of emotions

The classification of affective semantics in images is a very challenging research direction that gains more and more attention in the research community. However, as an emerging topic, contributions remain relatively rare, and a lot of issues need to be addressed particularly concerning the three following fundamentals problems: emotion representation, image features used to represent emotions and classification schemes designed to handle the distinctive characteristics of emotions. Thus, we present in this paper two classification approaches based on the dimensional and discrete emotion models. Traditional and emotional image features are used as input of classifiers relying on neural networks and on the evidence theory whose interesting properties allow to handle the ambiguous and subjective nature of emotions as it has been brought to the fore by our experimental results.

[1]  Gareth J. F. Jones,et al.  Affect-based indexing and retrieval of films , 2005, MULTIMEDIA '05.

[2]  Junqing Yu,et al.  Video Affective Content Representation and Recognition Using Video Affective Tree and Hidden Markov Models , 2007, ACII.

[3]  Joonwhoan Lee,et al.  A study of the emotional evaluation models of color patterns based on the adaptive fuzzy system and the neural network , 2002 .

[4]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2009, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Philippe Smets,et al.  The Combination of Evidence in the Transferable Belief Model , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Xufa Wang,et al.  Emotion Semantics Image Retrieval: An Brief Overview , 2005, ACII.

[8]  Qianhua He,et al.  A survey on emotional semantic image retrieval , 2008, 2008 15th IEEE International Conference on Image Processing.

[9]  Hang-Bong Kang,et al.  Affective content detection using HMMs , 2003, ACM Multimedia.

[10]  Alberto Del Bimbo,et al.  Semantics in Visual Information Retrieval , 1999, IEEE Multim..

[11]  Changle Zhou,et al.  Content-Based Affective Image Classification and Retrieval Using Support Vector Machines , 2005, ACII.

[12]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[13]  Alan Hanjalic,et al.  Affective video content representation and modeling , 2005, IEEE Transactions on Multimedia.

[14]  Masafumi Hagiwara,et al.  An image retrieval system by impression words and specific object names - IRIS , 2002, Neurocomputing.

[15]  Sung-Bae Cho,et al.  A human-oriented image retrieval system using interactive genetic algorithm , 2002, IEEE Trans. Syst. Man Cybern. Part A.

[16]  Johannes Itten,et al.  The art of color , 1961 .

[17]  Cheng-Te Li,et al.  Emotion-based impressionism slideshow with automatic music accompaniment , 2007, ACM Multimedia.

[18]  Stefanie Nowak,et al.  Content-based mood classification for photos and music: a generic multi-modal classification framework and evaluation approach , 2008, MIR '08.