Critical features for the perception of emotion from gait.

Human observers readily recognize emotions expressed in body movement. Their perceptual judgments are based on simple movement features, such as overall speed, but also on more intricate posture and dynamic cues. The systematic analysis of such features is complicated due to the difficulty of considering the large number of potentially relevant kinematic and dynamic parameters. To identify emotion-specific features we motion-captured the neutral and emotionally expressive (anger, happiness, sadness, fear) gaits of 25 individuals. Body posture was characterized by average flexion angles, and a low-dimensional parameterization of the spatio-temporal structure of joint trajectories was obtained by approximation with a nonlinear mixture model. Applying sparse regression, we extracted critical emotion-specific posture and movement features, which typically depended only on a small number of joints. The features we extracted from the motor behavior closely resembled features that were critical for the perception of emotion from gait, determined by a statistical analysis of classification and rating judgments of 21 observers presented with avatars animated with the recorded movements. The perceptual relevance of these features was further supported by another experiment showing that artificial walkers containing only the critical features induced high-level after-effects matching those induced by adaptation with natural emotional walkers.

[1]  W. Wundt Grundriss der Psychologie , 1896 .

[2]  N. Bull The attitude theory of emotion. , 1951, International record of medicine and general practice clinics.

[3]  H. Schlosberg Three dimensions of emotion. , 1954, Psychological review.

[4]  E. Gellhorn MOTION AND EMOTION: THE ROLE OF PROPRIOCEPTION IN THE PHYSIOLOGY AND PATHOLOGY OF THE EMOTIONS. , 1964, Psychological review.

[5]  P. Ekman Differential communication of affect by head and body cues. , 1965, Journal of personality and social psychology.

[6]  C. Osgood Dimensionality of the semantic space for communication via facial expressions. , 1966, Scandinavian journal of psychology.

[7]  C. Osgood Dimensionality of the semantic space for communication via facial expressions. , 1966, Scandinavian journal of psychology.

[8]  P. Ekman,et al.  Head and body cues in the judgment of emotion: a reformulation. , 1967, Perceptual and motor skills.

[9]  P. Ekman,et al.  Pan-Cultural Elements in Facial Displays of Emotion , 1969, Science.

[10]  P. Ekman,et al.  The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding , 1969 .

[11]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[12]  P. Ekman,et al.  Constants across cultures in the face and emotion. , 1971, Journal of personality and social psychology.

[13]  E. S. Bartlett,et al.  Patterns of Emotions: A New Analysis of Anxiety and Depression , 1972 .

[14]  C. Izard CHAPTER SIX – A Dimensional and Discrete Emotions Investigation of the Subjective Experience of Emotion* , 1972 .

[15]  P. Machotka,et al.  Messages of the Body , 1976 .

[16]  R. Rosenthal,et al.  Encoding and decoding nonverbal cues of emotion. , 1975, Journal of personality and social psychology.

[17]  J. N. Bassili Facial motion in the perception of faces and of emotional expression. , 1978, Journal of experimental psychology. Human perception and performance.

[18]  J. N. Bassili Facial motion in the perception of faces and of emotional expression. , 1978 .

[19]  Y. Chan,et al.  The least squares estimation of time delay and its use in signal detection , 1978 .

[20]  J. N. Bassili Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. , 1979, Journal of personality and social psychology.

[21]  R. D. Walk,et al.  Emotion and dance in dynamic light displays , 1984 .

[22]  J. Dovidio,et al.  Power, Dominance, and Nonverbal Behavior , 1985 .

[23]  N. Henley,et al.  The Nonverbal Semantics of Power and Gender: A Perceptual Study , 1985 .

[24]  K. Scherer,et al.  Cues and channels in emotion recognition. , 1986 .

[25]  J. Montepare,et al.  The identification of emotions from gait information , 1987 .

[26]  N. Frijda The laws of emotion. , 1988, The American psychologist.

[27]  S. Sogon,et al.  Identification of Emotion from Body Movements: A Cross-Cultural Study of Americans and Japanese , 1989 .

[28]  M. D. Meijer The contribution of general features of body movement to the attribution of emotions , 1989 .

[29]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[30]  A. J. Fridlund Sociality of Solitary Smiling: Potentiation by an Implicit Audience , 1991 .

[31]  M. D. Meijer,et al.  The attribution of aggression and grief to body movements: The effect of sex-stereotypes. , 1991 .

[32]  G. Mather,et al.  Low-level visual processing of biological motion , 1992, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[33]  John J. Magee,et al.  Categorical perception of facial expressions , 1992, Cognition.

[34]  Lester M. Hyman,et al.  Which are the stimuli in facial displays of anger and happiness? Configurational bases of emotion recognition. , 1992 .

[35]  P. Ekman Are there basic emotions? , 1992, Psychological review.

[36]  Alan J. Fridlund,et al.  Audience effects in affective imagery: Replication and extension to dysphoric imagery , 1992 .

[37]  Jeff T. Larsen,et al.  The psychophysiology of emotion. , 1993 .

[38]  C. Vaughan,et al.  Phasic behavior of EMG signals during gait: Use of multivariate statistics. , 1993, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[39]  M. Farah,et al.  Parts and Wholes in Face Recognition , 1993, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[40]  M. Farah,et al.  Parts and Wholes in Face Recognition , 1993, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[41]  W. Dittrich Action Categories and the Perception of Biological Motion , 1993, Perception.

[42]  J. Bortz Statistik: Fur Sozialwissenschaftler , 1993 .

[43]  Bennett I. Bertenthal,et al.  Global Processing of Biological Motions , 1994 .

[44]  S. Carey,et al.  Are faces perceived as configurations more by adults than by children , 1994 .

[45]  J. Hoogstraten,et al.  Head position and spinal position as determinants of perceived emotional state. , 1995, Perceptual and motor skills.

[46]  Ken-ichi Anjyo,et al.  Fourier principles for emotion-based human figure animation , 1995, SIGGRAPH.

[47]  Lance Williams,et al.  Motion signal processing , 1995, SIGGRAPH.

[48]  V. Bruce,et al.  Face processing: Human perception and principal components analysis , 1996, Memory & cognition.

[49]  S. Lea,et al.  Perception of Emotion from Dynamic Point-Light Displays Represented in Dance , 1996, Perception.

[50]  F. Hesse,et al.  Relative effectiveness and validity of mood induction procedures : a meta-analysis , 1996 .

[51]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[52]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[53]  J F Soechting,et al.  Matching object size by controlling finger span and hand shape. , 1997, Somatosensory & motor research.

[54]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[55]  H. Abdi,et al.  Principal Component and Neural Network Analyses of Face Images: What Can Be Generalized in Gender Classification? , 1997, Journal of mathematical psychology.

[56]  D. Massaro,et al.  Featural evaluation, integration, and judgment of facial affect. , 1997, Journal of experimental psychology. Human perception and performance.

[57]  L. A. Merkle,et al.  Using factor analysis to identify neuromuscular synergies during treadmill walking , 1998, Journal of Neuroscience Methods.

[58]  Wenjiang J. Fu Penalized Regressions: The Bridge versus the Lasso , 1998 .

[59]  Constance S. Royden,et al.  Motion perception , 1998 .

[60]  H. Wallbott Bodily expression of emotion , 1998 .

[61]  M. Golubitsky,et al.  Symmetry in locomotor central pattern generators and animal gaits , 1999, Nature.

[62]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[63]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[64]  Natasha Loder,et al.  Journal under attack over controversial paper on GM food , 1999, Nature.

[65]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[66]  Vladimir Vapnik,et al.  An overview of statistical learning theory , 1999, IEEE Trans. Neural Networks.

[67]  M. Shiffrar,et al.  Subconfigurations of the human form in the perception of biological motion displays. , 1999, Acta psychologica.

[68]  Michael J. Black,et al.  Parameterized Modeling and Recognition of Activities , 1999, Comput. Vis. Image Underst..

[69]  J. Montepare,et al.  The Use of Body Movements and Gestures as Cues to Emotions in Younger and Older Adults , 1999 .

[70]  A. Young,et al.  A principal component analysis of facial expressions , 2001, Vision Research.

[71]  Frédéric Gosselin,et al.  Bubbles: a technique to reveal the use of information in recognition tasks , 2001, Vision Research.

[72]  A. O'Toole,et al.  Prototype-referenced shape encoding revealed by high-level aftereffects , 2001, Nature Neuroscience.

[73]  Armin Bruderlin,et al.  Perceiving affect from arm movement , 2001, Cognition.

[74]  Karen L. Schmidt,et al.  Human facial expressions as adaptations: Evolutionary questions in facial expression research. , 2001, American journal of physical anthropology.

[75]  P. Beek,et al.  Coordination Between Arm and Leg Movements During Locomotion , 2001, Journal of motor behavior.

[76]  M. Lappe,et al.  Measurement of generalization fields for the recognition of biological motion , 2002, Vision Research.

[77]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[78]  I. Jolliffe Principal Component Analysis , 2002 .

[79]  Marco Santello,et al.  Patterns of Hand Motion during Grasping and the Influence of Sensory Guidance , 2002, The Journal of Neuroscience.

[80]  Sung-Bae Cho,et al.  Estimating the efficiency of recognizing gender and affect from biological motion , 2002, Vision Research.

[81]  Marian Stewart Bartlett,et al.  Face recognition by independent component analysis , 2002, IEEE Trans. Neural Networks.

[82]  Laurence T Maloney,et al.  Illuminant estimation as cue combination. , 2002, Journal of vision.

[83]  Laurence T Maloney,et al.  Illuminant estimation as cue combination. , 2002, Journal of vision.

[84]  Miguel P Eckstein,et al.  Classification images: a tool to analyze visual strategies. , 2002, Journal of vision.

[85]  Ole Winther,et al.  Mean-Field Approaches to Independent Component Analysis , 2002, Neural Computation.

[86]  Andrzej Cichocki,et al.  Adaptive Blind Signal and Image Processing - Learning Algorithms and Applications , 2002 .

[87]  A. O'Toole,et al.  Recognizing moving faces: a psychological and neural synthesis , 2002, Trends in Cognitive Sciences.

[88]  N. Troje Decomposing biological motion: a framework for analysis and synthesis of human gait patterns. , 2002, Journal of vision.

[89]  N. Troje Decomposing biological motion: a framework for analysis and synthesis of human gait patterns. , 2002, Journal of vision.

[90]  M. Webster,et al.  Adaptation to natural facial categories , 2002 .

[91]  J. Duysens,et al.  Adaptations in arm movements for added mass to wrist or ankle during walking , 2002, Experimental Brain Research.

[92]  K. Scherer,et al.  Handbook of affective sciences. , 2003 .

[93]  M. Sawada,et al.  Expression of Emotions in Dance: Relation between Arm Movement Characteristics and Emotion , 2003, Perceptual and motor skills.

[94]  K. Scherer,et al.  Vocal expression of emotion. , 2003 .

[95]  Mark D. Plumbley,et al.  BLIND SEPARATION OF POSITIVE SOURCES USING NON-NEGATIVE PC A , 2003 .

[96]  Mark D. Plumbley,et al.  BLIND SEPARATION OF POSITIVE SOURCES USING NON-NEGATIVE PC A , 2003 .

[97]  Christopher L. Vaughan,et al.  Fundamental patterns of bilateral muscle activity in human locomotion , 1995, Biological Cybernetics.

[98]  Martin A. Giese,et al.  Morphable Models for the Analysis and Synthesis of Complex Motion Patterns , 2000, International Journal of Computer Vision.

[99]  F. Lacquaniti,et al.  Five basic muscle activation patterns account for muscle activity during human locomotion , 2004, The Journal of physiology.

[100]  Mark C. Coulson Attributing Emotion to Static Body Postures: Recognition Accuracy, Confusions, and Viewpoint Dependence , 2004 .

[101]  A. Young,et al.  Emotion Perception from Dynamic and Static Body Expressions in Point-Light and Full-Light Displays , 2004, Perception.

[102]  Martin A. Giese,et al.  Trajectory synthesis by hierarchical spatio-temporal correspondence: comparison of different methods , 2005, APGV '05.

[103]  F. Lacquaniti,et al.  Coordination of Locomotion with Voluntary Movements in Humans , 2005, The Journal of Neuroscience.

[104]  Marina Pavlova,et al.  Perceived Dynamics of Static Images Enables Emotional Attribution , 2005, Perception.

[105]  Emilio Bizzi,et al.  Shared and specific muscle synergies in natural motor behaviors. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[106]  Zili Liu,et al.  Computing dynamic classification images from correlation maps. , 2006, Journal of vision.

[107]  Zili Liu,et al.  Computing dynamic classification images from correlation maps. , 2006, Journal of vision.

[108]  Andrea d'Avella,et al.  Matrix factorization algorithms for the identification of muscle synergies: evaluation on simulated and experimental data sets. , 2006, Journal of neurophysiology.

[109]  Mazyar Fallah,et al.  Adaptation of gender derived from biological motion , 2006, Nature Neuroscience.

[110]  K. Nakayama,et al.  Adaptation aftereffects in the perception of gender from biological motion. , 2006, Journal of vision.

[111]  Andrea Kleinsmith,et al.  Cross-cultural differences in recognizing affect from body posture , 2006, Interact. Comput..

[112]  Christopher Kirtley,et al.  Clinical Gait Analysis: Theory and Practice , 2006 .

[113]  B. Gelder Towards the neurobiology of emotional body language , 2006, Nature Reviews Neuroscience.

[114]  N. Meinshausen,et al.  Discussion: A tale of three cousins: Lasso, L2Boosting and Dantzig , 2007, 0803.3134.

[115]  A. Atkinson,et al.  Evidence for distinct contributions of form and motion information to the recognition of emotions from body gestures , 2007, Cognition.

[116]  Swann Pichon,et al.  Perceiving fear in dynamic body expressions , 2007, NeuroImage.

[117]  Garrison W. Cottrell,et al.  Identifying Emotion in Static Face Images , 2007 .

[118]  Thomas Hofmann,et al.  Blind source separation for over-determined delayed mixtures , 2007 .

[119]  Hillary Anger Elfenbein,et al.  Reading your Counterpart: The Benefit of Emotion Recognition Accuracy for Effectiveness in Negotiation , 2006 .

[120]  Lars Omlor,et al.  Extraction of spatio-temporal primitives of emotional body expressions , 2006, Neurocomputing.

[121]  Lucy S. Petro,et al.  Dynamics of Visual Information Integration in the Brain for Categorizing Facial Expressions , 2007, Current Biology.

[122]  Eric Luis Uhlmann,et al.  Can an Angry Woman Get Ahead? , 2008, Psychological science.

[123]  Albert Mukovskiy,et al.  Selforganization of character behavior by mixing of learned movement primitives , 2008, VMV.

[124]  Lars Omlor,et al.  Lateral asymmetry of bodily emotion expression , 2008, Current Biology.

[125]  W. Richards,et al.  Perception as Bayesian Inference , 2008 .

[126]  Richard M. Lipkin,et al.  Adaptation across the Cortical Hierarchy: Low-Level Curve Adaptation Affects High-Level Facial-Expression Judgments , 2008, The Journal of Neuroscience.

[127]  Shimon Edelman,et al.  Metrics of the perception of body movement. , 2008, Journal of vision.

[128]  Emily D Grossman,et al.  Temporal "Bubbles" reveal key features for point-light biological motion perception. , 2008, Journal of vision.

[129]  Lars Omlor,et al.  Features in the Recognition of Emotions from Dynamic Bodily Expression , 2009 .

[130]  R. Vogels,et al.  Functional differentiation of macaque visual temporal cortical neurons using a parametric action space. , 2009, Cerebral cortex.

[131]  Kris McDaniel,et al.  Parts and Wholes , 2010 .

[132]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[133]  Shigeru Akamatsu,et al.  Dynamic Properties Influence the Perception of Facial Expressions , 2001, Perception.