Taking the Hidden Route: Deep Mapping of Affect via 3D Neural Networks

In this note we address the problem of providing a fast, automatic, and coarse processing of the early mapping from emotional facial expression stimuli to the basic continuous dimensions of the core affect representation of emotions, namely valence and arousal. Taking stock of results in affective neuroscience, such mapping is assumed to be the earliest stage of a complex unfolding of processes that eventually entail detailed perception and emotional reaction involving the proper body.

[1]  C. Darwin.  The Expression of the Emotions in Man and Animals , .

[2]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[3]  Kenji Mase,et al.  Recognition of Facial Expression from Optical Flow , 1991 .

[4]  S. Paradiso.  The Emotional Brain: The Mysterious Underpinnings of Emotional Life , 1998 .

[5]  E. Vesterinen,et al.  Affective Computing , 2009, Encyclopedia of Biometrics.

[6]  R. Adolphs.  Recognizing emotion from facial expressions: psychological and neurological mechanisms. , 2002, Behavioral and cognitive neuroscience reviews.

[7]  R. Adolphs.  Neural systems for recognizing emotion , 2002, Current Opinion in Neurobiology.

[8]  J. Russell.  Core affect and the psychological construction of emotion. , 2003, Psychological review.

[9]  Zhigang Deng,et al.  Analysis of emotion recognition using facial expressions, speech and multimodal information , 2004, ICMI '04.

[10]  T. Dalgleish.  The emotional brain , 2004, Nature Reviews Neuroscience.

[11]  A. Goldman,et al.  Simulationist models of face-based emotion recognition , 2005, Cognition.

[12]  P. Ekman,et al.  What the face reveals : basic and applied studies of spontaneous expression using the facial action coding system (FACS) , 2005 .

[13]  K. Scherer,et al.  The World of Emotions is not Two-Dimensional , 2007, Psychological science.

[14]  Stefano Fusi,et al.  Emotion, cognition, and mental state representation in amygdala and prefrontal cortex. , 2010, Annual review of neuroscience.

[15]  Marta I. Garrido,et al.  Functional Evidence for a Dual Route to Amygdala , 2012, Current Biology.

[16]  Björn W. Schuller,et al.  Categorical and dimensional affect analysis in continuous input: Current trends and future directions , 2013, Image Vis. Comput..

[17]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[18]  W. Sommer,et al.  Facial EMG Responses to Emotional Expressions Are Related to Emotion Perception Ability , 2014, PloS one.

[19]  Giuseppe Boccignone,et al.  Affective Facial Expression Processing via Simulation: A Probabilistic Model , 2014, Biologically Inspired Cognitive Architectures.

[20]  Sidney K. D'Mello,et al.  A Review and Meta-Analysis of Multimodal Affect Detection Systems , 2015, ACM Comput. Surv..

[21]  Andrea Cavallaro,et al.  Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Anton Nijholt,et al.  The MAHNOB Mimicry Database: A database of naturalistic human interactions , 2015, Pattern Recognit. Lett..

[23]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[24]  Ralph Adolphs,et al.  Neuropsychology: How Many Emotions Are There? , 2015, Current Biology.

[25]  P. Niedenthal,et al.  Fashioning the Face: Sensorimotor Simulation Contributes to Facial Expression Recognition , 2016, Trends in Cognitive Sciences.

[26]  Thomas S. Huang,et al.  How deep neural networks can improve emotion recognition on video data , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[27]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[28]  Emad Barsoum,et al.  Emotion recognition in the wild from videos using images , 2016, ICMI.

[29]  Giuliano Grossi,et al.  Virtual EMG via Facial Video Analysis , 2017, ICIAP.

[30]  Raffaella Lanzarotti,et al.  AMHUSE: a multimodal dataset for HUmour SEnsing , 2017, ICMI.

[31]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).