Fuzzy-GIST for emotion recognition in natural scene images

Emotion modeling evoked by natural scenes is challenging issue. In this paper, we propose a novel scheme for analyzing the emotion reflected by a natural scene, considering the human emotional status. Based on the concept of original GIST, we developed the fuzzy-GIST to build the emotional feature space. According to the relationship between emotional factors and the characters of image, L*C*H* color and orientation information are chosen to study the relationship between human's low level emotions and image characteristics. And it is realized that we need to analyze the visual features at semantic level, so we incorporate the fuzzy concept to extract features with semantic meanings. Moreover, we treat emotional electroencephalography (EEG) using the fuzzy logic based on possibility theory rather than widely used conventional probability theory to generate the semantic feature of the human emotions. Fuzzy-GIST consists of both semantic visual information and linguistic EEG feature, it is used to represent emotional gist of a natural scene in a semantic level. The emotion evoked by an image is predicted from fuzzy-GIST by using a support vector machine, and the mean opinion score (MOS) is used for performance evaluation for the proposed scheme. The experiments results show that positive and negative emotions can be recognized with high accuracy for a given dataset.

[1]  Li Xu,et al.  An emotion-based approach to decision making and self learning in autonomous robot control , 2004, Fifth World Congress on Intelligent Control and Automation (IEEE Cat. No.04EX788).

[2]  P. Lang.  International affective picture system (IAPS) : affective ratings of pictures and instruction manual , 2005 .

[3]  Alberto Del Bimbo,et al.  Semantics in Visual Information Retrieval , 1999, IEEE Multim..

[4]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[5]  Erol Basar,et al.  Emotional face expressions are differentiated with brain oscillations. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[6]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[7]  Qing Zhang,et al.  Analysis of positive and negative emotions in natural scene using brain activity and GIST , 2009, Neurocomputing.

[8]  R. Held,et al.  MOVEMENT-PRODUCED STIMULATION IN THE DEVELOPMENT OF VISUALLY GUIDED BEHAVIOR. , 1963, Journal of comparative and physiological psychology.

[9]  L. Ou,et al.  A study of colour emotion and colour preference. Part I: Colour emotions for single colours , 2004 .

[10]  Rosalind W. Picard.  Affective Computing , 1997 .

[11]  Minho Lee,et al.  Modeling of recycling oxic and anoxic treatment system for swine wastewater using neural networks , 2000 .

[12]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[13]  James C. Bezdek,et al.  Fuzzy mathematics in pattern classification , 1973 .

[14]  Masayuki Inoue,et al.  MIC Interactive Dance System-an emotional interaction system , 2000, KES'2000. Fourth International Conference on Knowledge-Based Intelligent Engineering Systems and Allied Technologies. Proceedings (Cat. No.00TH8516).

[15]  Alberto Del Bimbo,et al.  Taking into Consideration Sports Semantic Annotation of Sports Videos Content-based Multimedia Indexing and Retrieval , 2002 .

[16]  Yu Ying-lin,et al.  Image Retrieval by Emotional Semantics: A Study of Emotional Space and Feature Extraction , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.