3D Fuzzy GIST to Analyze Emotional Features in Movies

In this paper, we propose a 3D fuzzy GIST to effectively describe the visual dynamic features related to the emotional characteristics in a movie clip. Unlike the previous fuzzy approaches, which use images, the proposed method employs movie clips and can dynamically extract the features to classify the emotional characteristics in a movie clip. The 3D fuzzy GIST based on 3D tensor data including L*C*H color (L: Lightness, C: Chroma, H: Hue) and orientation information in a movie clip can extract the visual dynamic features related to the emotional characteristics in a movie clip. The extracted visual dynamic features obtained by the proposed 3D fuzzy GIST are used as inputs to an adaptive neuro-fuzzy inference classifier. The classifier is provided with the mean opinion scores as the teaching signals. Experimental results show that the system with the proposed 3D fuzzy GIST feature extractor not only discriminates the positive emotional features from the negative ones but also identifies the changes of emotional features in movie clips successfully.

[1]  Alberto Del Bimbo,et al.  Semantics in Visual Information Retrieval , 1999, IEEE Multim..

[2]  Qing Zhang,et al.  Fuzzy-GIST for emotion recognition in natural scene images , 2009, 2009 IEEE 8th International Conference on Development and Learning.

[3]  Li Xu,et al.  An emotion-based approach to decision making and self learning in autonomous robot control , 2004, Fifth World Congress on Intelligent Control and Automation (IEEE Cat. No.04EX788).

[4]  Rosalind W. Picard.  Affective Computing , 1997 .

[5]  Erol Basar,et al.  Emotional face expressions are differentiated with brain oscillations. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[6]  Yu Ying-lin,et al.  Image Retrieval by Emotional Semantics: A Study of Emotional Space and Feature Extraction , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[7]  James C. Bezdek,et al.  Fuzzy mathematics in pattern classification , 1973 .

[8]  Qing Zhang,et al.  Emotion development system by interacting with human EEG and natural scene understanding , 2012, Cognitive Systems Research.

[9]  Antonio Torralba,et al.  Building the gist of a scene: the role of global image features in recognition. , 2006, Progress in brain research.

[10]  Alberto Del Bimbo,et al.  Taking into Consideration Sports Semantic Annotation of Sports Videos Content-based Multimedia Indexing and Retrieval , 2002 .

[11]  T. Strohmer,et al.  Gabor Analysis and Algorithms , 2012 .

[12]  L. Ou,et al.  A study of colour emotion and colour preference. Part I: Colour emotions for single colours , 2004 .

[13]  Qing Zhang,et al.  Autonomous emotion development using incremental modified adaptive neuro-fuzzy inference system , 2012, Neurocomputing.

[14]  E. Mizutani,et al.  Neuro-Fuzzy and Soft Computing-A Computational Approach to Learning and Machine Intelligence [Book Review] , 1997, IEEE Transactions on Automatic Control.

[15]  Didier Dubois,et al.  Possibility theory is not fully compositional! A comment on a short note by H.J. Greenberg , 1998, Fuzzy Sets Syst..

[16]  Qing Zhang,et al.  Analysis of positive and negative emotions in natural scene using brain activity and GIST , 2009, Neurocomputing.

[17]  Zhang Jian-chao,et al.  A new SVM based emotional classification of image , 2005 .

[18]  D. Heeger,et al.  Reliability of cortical activity during natural stimulation , 2010, Trends in Cognitive Sciences.