Emotional tagging of videos by exploring multiple emotions' coexistence

Videos may induce users' mixture emotions. Most present emotional tagging research ignore the phenomena of multiple emotions' coexistence and mutual exclusion. In this paper, we propose a novel emotional tagging approach by exploring multiple emotion's relations. First, several visual and audio features are extracted from videos. Second, support vector machines are used as the classifiers to get the measurements of emotional tags. Then, a Bayesian network is adopted to learn the relationships among emotional tags. After that, the Bayesian network is used to infer the video tags combining the measurements obtained by support vector machines. Experiments on a dataset of 72 affective videos demonstrate the effectiveness of our approach.

[1]  Bing Li,et al.  Horror movie scene recognition based on emotional perception , 2010, 2010 IEEE International Conference on Image Processing.

[2]  Loong Fah Cheong,et al.  Affective understanding in film , 2006, IEEE Trans. Circuits Syst. Video Technol..

[3]  Mohammad Soleymani,et al.  A Bayesian framework for video affective representation , 2009, 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops.

[4]  Mohammad Soleymani,et al.  Corpus Development for Affective Video Indexing , 2012, IEEE Transactions on Multimedia.

[5]  Shiliang Zhang,et al.  Affective MTV analysis based on arousal and valence features , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[6]  Chris H. Q. Ding,et al.  Analysis of gene expression profiles: class discovery and leaf ordering , 2002, RECOMB '02.

[7]  Shiliang Zhang,et al.  Affective Visualization and Retrieval for Music Video , 2010, IEEE Transactions on Multimedia.

[8]  J. A. Russel .nto faces: Resurrecting a dimensional-contextual perspective , 2007 .

[9]  Craig A. Smith,et al.  Dimensions of appraisal and physiological response in emotion. , 1989, Journal of personality and social psychology.

[10]  I. Mechelen,et al.  The co-occurrence of emotions in daily life: A multilevel approach , 2005 .

[11]  P. Philippot Inducing and assessing differentiated emotion-feeling states in the laboratory. , 1993, Cognition & emotion.

[12]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[13]  D. Cicchetti Emotion and Adaptation , 1993 .

[14]  J. Gross,et al.  Emotion elicitation using films , 1995 .

[15]  Ling-Yu Duan,et al.  Hierarchical movie affective content analysis based on arousal and valence features , 2008, ACM Multimedia.

[16]  Peter Y. K. Cheung,et al.  A Novel Probabilistic Approach to Modeling the Pleasure-Arousal-Dominance Content of the Video based on "Working Memory" , 2007, International Conference on Semantic Computing (ICSC 2007).

[17]  Xufa Wang,et al.  Emotional Semantic Detection from Multimedia: A Brief Overview , 2011 .

[18]  Yuri Miyamoto,et al.  Culture and mixed emotions: co-occurrence of positive and negative emotions in Japan and the United States. , 2010, Emotion.

[19]  Qiang Ji,et al.  Efficient Structure Learning of Bayesian Networks using Constraints , 2011, J. Mach. Learn. Res..

[20]  Alan Hanjalic,et al.  Affective video content representation and modeling , 2005, IEEE Transactions on Multimedia.

[21]  Xiangjian He,et al.  Using Scripts for Affective Content Retrieval , 2010, PCM.

[22]  Hang-Bong Kang,et al.  Affective content detection using HMMs , 2003, ACM Multimedia.

[23]  Kiyoharu Aizawa,et al.  Determination of emotional content of video clips by low-level audiovisual features , 2011, Multimedia Tools and Applications.

[24]  Kiyoharu Aizawa,et al.  Affective Audio-Visual Words and Latent Topic Driving Model for Realizing Movie Affective Scene Classification , 2010, IEEE Transactions on Multimedia.

[25]  Noriko Tomuro,et al.  Clustering Using Feature Domain Similarity to Discover Word Senses for Adjectives , 2007 .

[26]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[27]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[28]  Ishwar K. Sethi,et al.  Classification of general audio data for content-based retrieval , 2001, Pattern Recognit. Lett..

[29]  R. Lazarus Emotion and Adaptation , 1991 .

[30]  Riccardo Leonardi,et al.  Emotional identity of movies , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[31]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[32]  A. Schaefer,et al.  Please Scroll down for Article Cognition & Emotion Assessing the Effectiveness of a Large Database of Emotion-eliciting Films: a New Tool for Emotion Researchers , 2022 .

[33]  Peter Y. K. Cheung,et al.  Affective Level Video Segmentation by Utilizing the Pleasure-Arousal-Dominance Information , 2008, IEEE Transactions on Multimedia.