Recognizing emotions expressed by body pose: A biologically inspired neural model

Research into the visual perception of human emotion has traditionally focused on the facial expression of emotions. Recently researchers have turned to the more challenging field of emotional body language, i.e. emotion expression through body pose and motion. In this work, we approach recognition of basic emotional categories from a computational perspective. In keeping with recent computational models of the visual cortex, we construct a biologically plausible hierarchy of neural detectors, which can discriminate seven basic emotional states from static views of associated body poses. The model is evaluated against human test subjects on a recent set of stimuli manufactured for research on emotional body language.

[1]  O. Andreassen,et al.  Mice Deficient in Cellular Glutathione Peroxidase Show Increased Vulnerability to Malonate, 3-Nitropropionic Acid, and 1-Methyl-4-Phenyl-1,2,5,6-Tetrahydropyridine , 2000, The Journal of Neuroscience.

[2]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[3]  T. Dalgleish Basic Emotions , 2004 .

[4]  Guang-Zhong Yang,et al.  Designing a Posture Analysis System with Hardware Implementation , 2007, J. VLSI Signal Process..

[5]  P. Downing,et al.  The neural basis of visual body perception , 2007, Nature Reviews Neuroscience.

[6]  B. Gelder Towards the neurobiology of emotional body language , 2006, Nature Reviews Neuroscience.

[7]  Kunihiko Fukushima,et al.  Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position , 1980, Biological Cybernetics.

[8]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[9]  Philip J. Benson,et al.  Categorical Perception of Facial Expressions: Categories and their Internal Structure , 1997 .

[10]  T. Poggio,et al.  Cognitive neuroscience: Neural mechanisms for the recognition of biological movements , 2003, Nature Reviews Neuroscience.

[11]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[12]  Sabine Kastner,et al.  Interactive report A control of the processing of neutral and emotional stimuli , 2002 .

[13]  A. Ortony,et al.  What's basic about basic emotions? , 1990, Psychological review.

[14]  Mun Wai Lee,et al.  A model-based approach for estimating human 3D poses in static images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  M. Eimer,et al.  An ERP study on the time course of emotional face processing , 2002, Neuroreport.

[16]  R. Desimone,et al.  Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. , 1981, Journal of neurophysiology.

[17]  T. Gawne,et al.  Responses of primate visual cortical neurons to stimuli presented by flash, saccade, blink, and external darkening. , 2002, Journal of neurophysiology.

[18]  Beat Fasel,et al.  Automatic facial expression analysis: a survey , 2003, Pattern Recognit..

[19]  N. Kanwisher,et al.  A Cortical Area Selective for Visual Processing of the Human Body , 2001, Science.

[20]  P. Lennie Receptive fields , 2003, Current Biology.

[21]  Béatrice de Gelder,et al.  Seeing Fearful Body Language Overcomes Attentional Deficits in Patients with Neglect , 2007, Journal of Cognitive Neuroscience.

[22]  T. Gawne,et al.  Responses of primate visual cortical V4 neurons to simultaneously presented stimuli. , 2002, Journal of neurophysiology.

[23]  Davide Anguita,et al.  Improved neural network for SVM learning , 2002, IEEE Trans. Neural Networks.

[24]  Nicu Sebe,et al.  Towards authentic emotion recognition , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[25]  N. Hadjikhani,et al.  Fear fosters flight: a mechanism for fear contagion when perceiving emotion expressed by a whole body. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[26]  P. Ekman Facial expression and emotion. , 1993, The American psychologist.

[27]  Bradford Z. Mahon,et al.  Action-Related Properties Shape Object Representations in the Ventral Stream , 2007, Neuron.

[28]  Garrison W. Cottrell,et al.  Representing Face Images for Emotion Classification , 1996, NIPS.

[29]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[30]  P A Salin,et al.  Corticocortical connections in the visual system: structure and function. , 1995, Physiological reviews.

[31]  Swann Pichon,et al.  Perceiving fear in dynamic body expressions , 2007, NeuroImage.

[32]  Tjeerd Jellema,et al.  Neural representations of perceived bodily actions using a categorical frame of reference , 2006, Neuropsychologia.

[33]  R. Adolphs Neural systems for recognizing emotion , 2002, Current Opinion in Neurobiology.

[34]  P. Ekman An argument for basic emotions , 1992 .

[35]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[36]  S. Thorpe,et al.  Speed of processing in the human visual system , 1996, Nature.

[37]  A. T. Smith,et al.  Estimating receptive field size from fMRI data in human striate and extrastriate visual cortex. , 2001, Cerebral cortex.

[38]  John G. Taylor,et al.  The interaction of attention and emotion , 2005, Neural Networks.

[39]  L. Weiskrantz,et al.  Differential extrageniculostriate and amygdala responses to presentation of emotional faces in a cortically blind field. , 2001, Brain : a journal of neurology.

[40]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[41]  J. Hegdé,et al.  Selectivity for Complex Shapes in Primate Visual Area V2 , 2000, The Journal of Neuroscience.

[42]  Mike W Oram,et al.  The sensitivity of primate STS neurons to walking sequences and to the degree of articulation in static images. , 2006, Progress in brain research.

[43]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[44]  H. Meeren,et al.  Rapid perceptual integration of facial expression and emotional body language. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[45]  C. Izard Basic emotions, relations among emotions, and emotion-cognition relations. , 1992, Psychological review.

[46]  D. Lehmann,et al.  Rapid emotional face processing in the human right and left brain hemispheres: an ERP study. , 1999, Neuroreport.

[47]  R. Dolan,et al.  Effects of Attention and Emotion on Face Processing in the Human Brain An Event-Related fMRI Study , 2001, Neuron.

[48]  Thomas Serre,et al.  Robust Object Recognition with Cortex-Like Mechanisms , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  J. G. Taylor,et al.  Emotion recognition in human-computer interaction , 2005, Neural Networks.

[50]  Leslie G. Ungerleider,et al.  Neural processing of emotional faces requires attention , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[51]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[52]  J. Russell Is there universal recognition of emotion from facial expression? A review of the cross-cultural studies. , 1994, Psychological bulletin.

[53]  A. Dale,et al.  Functional analysis of primary visual cortex (V1) in humans. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[54]  Mark H. Johnson,et al.  Categorical Perception of Facial Expressions by 7-Month-Old Infants , 2001, Perception.

[55]  J. Stainer,et al.  The Emotions , 1882, Nature.

[56]  J. Stekelenburg,et al.  The neural correlates of perceiving human bodies: an ERP study on the body-inversion effect , 2004, Neuroreport.

[57]  David A. Leopold,et al.  Physiologically inspired neural model for the encoding of face spaces , 2005, Neurocomputing.

[58]  P. Ekman Universal facial expressions of emotion. , 1970 .

[59]  D. Perrett,et al.  Responses of Anterior Superior Temporal Polysensory (STPa) Neurons to Biological Motion Stimuli , 1994, Journal of Cognitive Neuroscience.

[60]  J A Beintema,et al.  Perception of biological motion without local image motion , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[61]  D J Field,et al.  Relations between the statistics of natural images and the response properties of cortical cells. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[62]  François Brémond,et al.  Applying 3D human model in a posture recognition system , 2006, Pattern Recognit. Lett..

[63]  D. Perrett,et al.  Categorical Perception of Morphed Facial Expressions , 1996 .

[64]  Thomas Serre,et al.  A feedforward architecture accounts for rapid categorization , 2007, Proceedings of the National Academy of Sciences.

[65]  H. Bourlard,et al.  Auto-association by multilayer perceptrons and singular value decomposition , 1988, Biological Cybernetics.

[66]  Tomaso Poggio,et al.  Intracellular measurements of spatial integration and the MAX operation in complex cells of the cat primary visual cortex. , 2004, Journal of neurophysiology.

[67]  G. Cottrell,et al.  EMPATH: A Neural Network that Categorizes Facial Expressions , 2002, Journal of Cognitive Neuroscience.

[68]  N. Logothetis,et al.  Shape representation in the inferior temporal cortex of monkeys , 1995, Current Biology.

[69]  A. Ohman,et al.  Emotion drives attention: detecting the snake in the grass. , 2001, Journal of experimental psychology. General.