AV+EC 2015: The First Affect Recognition Challenge Bridging Across Audio, Video, and Physiological Data

We present the first Audio-Visual+ Emotion recognition Challenge and workshop (AV+EC 2015) aimed at comparison of multimedia processing and machine learning methods for automatic audio, visual and physiological emotion analysis. This is the 5th event in the AVEC series, but the very first Challenge that bridges across audio, video and physiological data. The goal of the Challenge is to provide a common benchmark test set for multimodal information processing and to bring together the audio, video and physiological emotion recognition communities, to compare the relative merits of the three approaches to emotion recognition under well-defined and strictly comparable conditions and establish to what extent fusion of the approaches is possible and beneficial. This paper presents the challenge, the dataset and the performance of the baseline system.

[1]  Maja Pantic,et al.  The first facial expression recognition and analysis challenge , 2011, Face and Gesture 2011.

[2]  Björn W. Schuller,et al.  The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing , 2016, IEEE Transactions on Affective Computing.

[3]  Björn W. Schuller,et al.  Introducing CURRENNT: the munich open-source CUDA recurrent neural network toolkit , 2015, J. Mach. Learn. Res..

[4]  C. Nickerson.  A note on a concordance correlation coefficient to evaluate reproducibility , 1997 .

[5]  Rosalind W. Picard,et al.  Quantitative analysis of wrist electrodermal activity during sleep. , 2014, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[6]  Jean-Philippe Thiran,et al.  Prediction of asynchronous dimensional emotion ratings from audiovisual and physiological data , 2015, Pattern Recognit. Lett..

[7]  Fernando De la Torre,et al.  Supervised Descent Method and Its Applications to Face Alignment , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Nathan Halko,et al.  An Algorithm for the Principal Component Analysis of Large Data Sets , 2010, SIAM J. Sci. Comput..

[9]  Fabien Ringeval,et al.  On the Influence of Emotional Feedback on Emotion Awareness and Gaze Behavior , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[10]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[11]  K. Kroschel,et al.  Evaluation of natural emotions using self assessment manikins , 2005, IEEE Workshop on Automatic Speech Recognition and Understanding, 2005..

[12]  J. Fleiss,et al.  Intraclass correlations: uses in assessing rater reliability. , 1979, Psychological bulletin.

[13]  E. Miller.  Handbook of Social Psychology , 1946, Mental Health.

[14]  Michel F. Valstar,et al.  Local Gabor Binary Patterns from Three Orthogonal Planes for Automatic Facial Expression Recognition , 2013, 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction.

[15]  Fabien Ringeval,et al.  Emotion Recognition in the Wild: Incorporating Voice and Lip Activity in Multimodal Decision-Level Fusion , 2014, ICMI.

[16]  R. B. Knapp,et al.  Physiological signals and their use in augmenting emotion recognition for human-machine interaction , 2011 .

[17]  Lijun Yin,et al.  FERA 2015 - second Facial Expression Recognition and Analysis challenge , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[18]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[19]  Min Chen,et al.  AIWAC: affective interaction through wearable computing and cloud technology , 2015, IEEE Wireless Communications.

[20]  M. Dawson,et al.  The electrodermal system , 2007 .

[21]  Sravanti L. Sanivarapu.  Emotion , 2020, Indian journal of psychiatry.

[22]  Katherine B. Martin,et al.  Facial Action Coding System , 2015 .

[23]  Fabien Ringeval,et al.  Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[24]  Maja Pantic,et al.  AVEC 2014 – The Three Dimensional Affect and Depression Challenge , 2014 .

[25]  Shrikanth S. Narayanan,et al.  Robust Unsupervised Arousal Rating:A Rule-Based Framework withKnowledge-Inspired Vocal Features , 2014, IEEE Transactions on Affective Computing.

[26]  K. Scherer,et al.  Appraisal processes in emotion: Theory, methods, research. , 2001 .

[27]  L. Cronbach.  Coefficient alpha and the internal structure of tests , 1951 .

[28]  Rosalind W. Picard.  Affective media and wearables: surprising findings , 2014, ACM Multimedia.

[29]  Björn W. Schuller,et al.  Recent developments in openSMILE, the munich open-source multimedia feature extractor , 2013, ACM Multimedia.

[30]  L. Lin,et al.  A concordance correlation coefficient to evaluate reproducibility. , 1989, Biometrics.

[31]  Stanley Schachter,et al.  Chapter 18 – Cognition and Peripheralist—Centralist Controversies in Motivation and Emotion , 1975 .

[32]  Björn W. Schuller,et al.  AVEC 2014: 3D Dimensional Affect and Depression Recognition Challenge , 2014, AVEC '14.