Evaluation of Features and Combination Approaches for the Classification of Emotional Semantics in Images

Recognition of emotional semantics in images is a new and very challenging research direction that gains more and more attention in the research community. As an emerging topic, publications remains relatively rare and numerous issues need to be addressed. In this paper, we propose to investigate the efficiency of different types of features including low-level features and proposed semantic features for classification of emotional semantics in images. Moreover, we propose a new approach that combines different classifiers based on Dempster-Shafer’s theory of evidence, which has the ability to handle ambiguous and uncertain knowledge such as the properties of emotions. Experiments driven on the International Affective Picture System (IAPS) image databases, which is a common stimulus set frequently used in emotion psychology research, demonstrated that the proposed approach can achieve promising results.

[1]  James Ze Wang,et al.  Content-based image retrieval: approaches and trends of the new age , 2005, MIR '05.

[2]  Qianhua He,et al.  A survey on emotional semantic image retrieval , 2008, 2008 15th IEEE International Conference on Image Processing.

[3]  Changle Zhou,et al.  Content-Based Affective Image Classification and Retrieval Using Support Vector Machines , 2005, ACII.

[4]  Xufa Wang,et al.  Emotion Semantics Image Retrieval: An Brief Overview , 2005, ACII.

[5]  Philippe Smets,et al.  The Combination of Evidence in the Transferable Belief Model , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Nicu Sebe,et al.  Emotional valence categorization using holistic image features , 2008, 2008 15th IEEE International Conference on Image Processing.

[7]  Johannes Itten,et al.  The art of color , 1961 .

[8]  Yu Ying-lin,et al.  Image Retrieval by Emotional Semantics: A Study of Emotional Space and Feature Extraction , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[9]  Cheng-Te Li,et al.  Emotion-based impressionism slideshow with automatic music accompaniment , 2007, ACM Multimedia.

[10]  Arthur P. Dempster,et al.  A Generalization of Bayesian Inference , 1968, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[11]  Masafumi Hagiwara,et al.  An image retrieval system by impression words and specific object names - IRIS , 2002, Neurocomputing.

[12]  Alberto Del Bimbo,et al.  Semantics in Visual Information Retrieval , 1999, IEEE Multim..

[13]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[15]  Stefanie Nowak,et al.  Content-based mood classification for photos and music: a generic multi-modal classification framework and evaluation approach , 2008, MIR '08.

[16]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Gareth J. F. Jones,et al.  Affect-based indexing and retrieval of films , 2005, MULTIMEDIA '05.

[18]  Zhihong Zeng,et al.  A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.