Revisiting AVEC 2011 - An Information Fusion Architecture

Combining information from multiple sources is a vivid field of research. The problem of emotion recognition is inherently multi-modal. As automatic recognition of the emotional states is performed imperfectly by the single mode classifiers, its combination is crucial. In this work, the AVEC 2011 corpus is used to evaluate several machine learning techniques in the context of information fusion. In particular temporal integration of intermediate results combined with a reject option based on classifier confidences. The results for the modes are combined using a Markov random field that is designed to be able to tackle failures of individual channels.

[1]  Mário A. T. Figueiredo,et al.  Similarity-Based Clustering of Sequences Using Hidden Markov Models , 2003, MLDM.

[2]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[3]  Björn W. Schuller,et al.  AVEC 2011-The First International Audio/Visual Emotion Challenge , 2011, ACII.

[4]  C. K. Chow,et al.  On optimum recognition error and reject tradeoff , 1970, IEEE Trans. Inf. Theory.

[5]  Zheng Fang,et al.  Comparison of different implementations of MFCC , 2001 .

[6]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[7]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[8]  Hynek Hermansky,et al.  RASTA-PLP speech analysis technique , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[9]  J. Besag.  On the Statistical Analysis of Dirty Pictures , 1986 .

[10]  Gwen Littlewort,et al.  The computer expression recognition toolbox (CERT) , 2011, Face and Gesture 2011.

[11]  Günther Palm,et al.  Using Dempster-Shafer Theory in MCF Systems to Reject Samples , 2005, Multiple Classifier Systems.

[12]  Mario Vento,et al.  To reject or not to reject: that is the question-an answer in case of neural classifiers , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[13]  Maja Pantic,et al.  The SEMAINE corpus of emotionally coloured character interactions , 2010, 2010 IEEE International Conference on Multimedia and Expo.