Information Fusion VIA Optimized KECA with Application to Audio Emotion Recognition

As a recent proposed information fusion tool, Kernel Entropy Component Analysis (KECA) has attracted more attentions from the research communities of multimedia. It utilizes descriptor of entropy estimation and achieves improved performance for information fusion. However, KECA roughly reduces to sorting the importance of kernel eigenvectors by entropy instead of by variance as in Kernel Principal Components Analysis (KPCA), without extracting the optimal features retaining more entropy of the input data. In this paper, a novel approach Optimized Kernel Entropy Components Analysis (OKECA) is introduced to information fusion, which can be considered as an alternative method to KECA for information fusion. Since OKECA explicitly extracts the optimal features that retain most informative content, it leads to improving the final performance or classification accuracy. To demonstrate the effectiveness of the proposed solution, experiments are conducted on Ryerson Multimedia Lab (RML) and eNTERFACE emotion datasets. Experimental results show that the proposed solution outperforms the existing methods based on the similar principles, and the Deep Learning (DL) based method.

[1]  Oh-Wook Kwon,et al.  EMOTION RECOGNITION BY SPEECH SIGNAL , 2003 .

[2]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[3]  Wei Xiong,et al.  Stacked Convolutional Denoising Auto-Encoders for Feature Representation , 2017, IEEE Transactions on Cybernetics.

[4]  Manuele Bicego,et al.  Audio-Visual Event Recognition in Surveillance Video Sequences , 2007, IEEE Transactions on Multimedia.

[5]  Ling Guan,et al.  An investigation of speech-based human emotion recognition , 2004, IEEE 6th Workshop on Multimedia Signal Processing, 2004..

[6]  Ioannis Pitas,et al.  Automatic emotional speech classification , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[7]  James V. Stone Independent Component Analysis: A Tutorial Introduction , 2007 .

[8]  Alfréd Rényi,et al.  Foundations of Probability , 1971 .

[9]  Colin Fyfe,et al.  Kernel and Nonlinear Canonical Correlation Analysis , 2000, IJCNN.

[10]  Vince D. Calhoun,et al.  Canonical Correlation Analysis for Data Fusion and Group Inferences , 2010, IEEE Signal Processing Magazine.

[11]  Satosi Watanabe,et al.  Pattern Recognition: Human and Mechanical , 1985 .

[12]  Ling Guan,et al.  Multimodal Information Fusion of Audio Emotion Recognition Based on Kernel Entropy Component Analysis , 2012, 2012 IEEE International Symposium on Multimedia.

[13]  Ling Guan,et al.  Multimodal information fusion for selected multimedia applications , 2010, Int. J. Multim. Intell. Secur..

[14]  Ling Guan,et al.  Recognizing Human Emotional State From Audiovisual Signals* , 2008, IEEE Transactions on Multimedia.

[15]  Jian Yang,et al.  Generalized K-L transform based combined feature extraction , 2002, Pattern Recognit..

[16]  Jorge A. Balazs,et al.  Opinion Mining and Information Fusion: A survey , 2016, Inf. Fusion.

[17]  Nicole M. Russo,et al.  Musical experience shapes human brainstem encoding of linguistic pitch patterns , 2007, Nature Neuroscience.

[18]  G. Camps-Valls,et al.  Kernel Entropy Component Analysis for Remote Sensing Image Clustering , 2012, IEEE Geoscience and Remote Sensing Letters.

[19]  Wen Gao,et al.  Learning Affective Features With a Hybrid Deep Model for Audio–Visual Emotion Recognition , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[20]  Mohan S. Kankanhalli,et al.  Multimodal fusion for multimedia analysis: a survey , 2010, Multimedia Systems.

[21]  Klaus-Robert Müller,et al.  Analyzing Local Structure in Kernel-Based Learning: Explanation, Complexity, and Reliability Assessment , 2013, IEEE Signal Processing Magazine.

[22]  Ling Guan,et al.  Multimodal information fusion of audiovisual emotion recognition using novel information theoretic tools , 2013, ICME.

[23]  Robert Jenssen,et al.  Kernel Entropy Component Analysis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Chengjun Liu,et al.  A shape- and texture-based enhanced Fisher classifier for face recognition , 2001, IEEE Trans. Image Process..