Dissociable Neural Pathways Are Involved in the Recognition of Emotion in Static and Dynamic Facial Expressions

Facial expressions of emotion powerfully influence social behavior. The distributed network of brain regions thought to decode these social signals has been empirically defined using static, usually photographic, displays of such expressions. Facial emotional expressions are however highly dynamic signals that encode the emotion message in facial action patterns. This study sought to determine whether the encoding of facial expressions of emotion by static or dynamic displays is associated with different neural correlates for their decoding. We used positron emission tomography to compare patterns of brain activity in healthy men and women during the explicit judgment of emotion intensity in static and dynamic facial expressions of anger and happiness. Compared to judgments of spatial orientation for moving neutral facial expressions, the judgment of anger in dynamic expressions was associated with increased right-lateralized activity in the medial, superior, middle, and inferior frontal cortex and cerebellum, while judgments of happiness were associated with relative activation of the cuneus, temporal cortex, and the middle, medial, and superior frontal cortex. In contrast, the perception of anger or happiness in static facial expressions activated a motor, prefrontal, and parietal cortical network previously shown to be involved in motor imagery. The direct contrast of dynamic and static expressions indicated differential activation of visual area V5, superior temporal sulcus, periamygdaloid cortex, and cerebellum for dynamic angry expressions and differential activation of area V5, extrastriate cortex, brain stem, and middle temporal cortical activations for dynamic happy expressions. Thus, a distribution of neural activations is related to the analysis of emotion messages in the nearly constant biological motion of the face and differ for angry and happy expressions. Static displays of facial emotional expression may represent noncanonical stimuli that are processed for emotion content by mental strategies and neural events distinct from their more ecologically relevant dynamic counterparts.

[1]  B. Bertenthal,et al.  Does Perception of Biological Motion Rely on Specific Brain Regions? , 2001, NeuroImage.

[2]  M. Tarr,et al.  Activation of the middle fusiform 'face area' increases with expertise in recognizing novel objects , 1999, Nature Neuroscience.

[3]  Glyn W. Humphreys,et al.  Expression is computed separately from facial identity, and it is computed separately for moving and static faces: Neuropsychological evidence , 1993, Neuropsychologia.

[4]  R. Desimone Face-Selective Cells in the Temporal Cortex of Monkeys , 1991, Journal of Cognitive Neuroscience.

[5]  G. Johansson Visual perception of biological motion and a model for its analysis , 1973 .

[6]  M. Harries,et al.  Visual Processing of Faces in Temporal Cortex: Physiological Evidence for a Modular Organization and Possible Anatomical Correlates , 1991, Journal of Cognitive Neuroscience.

[7]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[8]  J. Neale,et al.  Emotion perception in schizophrenia: specific deficit or further evidence of generalized poor performance? , 1993, Journal of abnormal psychology.

[9]  K. Scherer,et al.  Studying the dynamics of emotional expression using synthesized facial muscle movements. , 2000, Journal of personality and social psychology.

[10]  D. Perrett,et al.  A differential neural response in the human amygdala to fearful and happy facial expressions , 1996, Nature.

[11]  P. Ekman Emotion in the human face , 1982 .

[12]  A. Young,et al.  Neuropsychology of fear and loathing , 2001 .

[13]  R. Adolphs,et al.  A Role for Somatosensory Cortices in the Visual Recognition of Emotion as Revealed by Three-Dimensional Lesion Mapping , 2000, The Journal of Neuroscience.

[14]  Simon R. Cherry,et al.  Improved Detection of Focal Cerebral Blood Flow Changes Using Three-Dimensional Positron Emission Tomography , 1993, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[15]  U. Dimberg,et al.  Unconscious Facial Reactions to Emotional Facial Expressions , 2000, Psychological science.

[16]  Leslie G. Ungerleider,et al.  The functional organization of human extrastriate cortex: a PET-rCBF study of selective attention to faces and locations , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[17]  R. Adolphs Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala , 1997 .

[18]  T Allison,et al.  ERPS EVOKED BY VIEWING FACIAL MOVEMENTS , 2000, Cognitive neuropsychology.

[19]  T. Allison,et al.  Temporal Cortex Activation in Humans Viewing Eye and Mouth Movements , 1998, The Journal of Neuroscience.

[20]  V. Barnett,et al.  Applied Linear Statistical Models , 1975 .

[21]  Håkan Fischer,et al.  Enhanced amygdala responses to emotional versus neutral schematic facial expressions , 2002, Neuroreport.

[22]  Demetri Terzopoulos,et al.  A physical model of facial tissue and muscle articulation , 1990, [1990] Proceedings of the First Conference on Visualization in Biomedical Computing.

[23]  M. Erb,et al.  Activation of Cortical and Cerebellar Motor Areas during Executed and Imagined Hand Movements: An fMRI Study , 1999, Journal of Cognitive Neuroscience.

[24]  Nirbhay N. Singh,et al.  Recognition of Facial Expressions of Emotion by Children with Attention-Deficit Hyperactivity Disorder , 1998, Behavior modification.

[25]  T. Allison,et al.  Social perception from visual cues: role of the STS region , 2000, Trends in Cognitive Sciences.

[26]  L. Parsons,et al.  Use of implicit motor imagery for visual shape discrimination as revealed by PET , 1995, Nature.

[27]  D C Hay,et al.  Movement, face processing and schizophrenia: evidence of a differential deficit in expression analysis. , 1994, The British journal of clinical psychology.

[28]  R. Tootell,et al.  Anatomical evidence for MT and additional cortical visual areas in humans. , 1995, Cerebral cortex.

[29]  J B Poline,et al.  Partially overlapping neural networks for real and imagined hand movements. , 2000, Cerebral cortex.

[30]  M. Jeannerod,et al.  Mental motor imagery: a window into the representational stages of action , 1995, Current Opinion in Neurobiology.

[31]  D. Perrett,et al.  Dissociable neural responses to facial expressions of sadness and anger. , 1999, Brain : a journal of neurology.

[32]  R. Blake,et al.  Brain Areas Involved in Perception of Biological Motion , 2000, Journal of Cognitive Neuroscience.

[33]  J. Decety,et al.  Neural mechanisms subserving the perception of human actions , 1999, Trends in Cognitive Sciences.

[34]  Alan C. Evans,et al.  Specific Involvement of Human Parietal Systems and the Amygdala in the Perception of Biological Motion , 1996, The Journal of Neuroscience.

[35]  Lester M. Hyman,et al.  Which are the stimuli in facial displays of anger and happiness? Configurational bases of emotion recognition. , 1992 .

[36]  R. Maddock,et al.  Impaired recognition of facial expressions of emotion in Alzheimer's disease. , 2002, The Journal of neuropsychiatry and clinical neurosciences.

[37]  R. Knight,et al.  Prefrontal modulation of visual processing in humans , 2000, Nature Neuroscience.

[38]  Martha J. Farah,et al.  Face perception and within-category discrimination in prosopagnosia , 1995, Neuropsychologia.

[39]  S. Rauch,et al.  A functional MRI study of human amygdala responses to facial expressions of fear versus anger. , 2001, Emotion.

[40]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[41]  T. Allison,et al.  Differential Sensitivity of Human Visual Cortex to Faces, Letterstrings, and Textures: A Functional Magnetic Resonance Imaging Study , 1996, The Journal of Neuroscience.

[42]  J. Mazziotta,et al.  A Noninvasive Positron Computed Tomography Technique Using Oxygen-15-Labeled Water for the Evaluation of Neurobehavioral Task Batteries , 1985, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[44]  A. Young,et al.  Recognition of facial emotion in nine individuals with bilateral amygdala damage , 1999, Neuropsychologia.

[45]  Jeffrey M. Zacks,et al.  Imagined transformations of bodies: an fMRI investigation , 1999, Neuropsychologia.

[46]  Edward T. Bullmore,et al.  A differential neural response to threatening and non-threatening negative facial expressions in paranoid and non-paranoid schizophrenics , 1999, Psychiatry Research: Neuroimaging.

[47]  S. Rauch,et al.  Response and Habituation of the Human Amygdala during Visual Processing of Facial Expression , 1996, Neuron.

[48]  E. Rolls The orbitofrontal cortex and reward. , 2000, Cerebral cortex.

[49]  K. Nakamura,et al.  The human amygdala plays an important role in gaze monitoring. A PET study. , 1999, Brain : a journal of neurology.

[50]  R. Gur,et al.  Emotion recognition deficit in schizophrenia: association with symptomatology and cognition , 2000, Biological Psychiatry.

[51]  Karl J. Friston,et al.  Cognitive Conjunction: A New Approach to Brain Activation Experiments , 1997, NeuroImage.

[52]  Klaus R. Scherer,et al.  Studying the dynamics of emotional expression using synthesized facial muscle movements. , 2000 .

[53]  E. Bullmore,et al.  Investigation of facial recognition memory and happy and sad facial expression perception: an fMRI study , 1998, Psychiatry Research: Neuroimaging.

[54]  P. Whalen Fear, Vigilance, and Ambiguity , 1998 .

[55]  E. Courchesne,et al.  Face processing occurs outside the fusiform 'face area' in autism: evidence from functional MRI. , 2001, Brain : a journal of neurology.

[56]  J. Borod,et al.  Perception of Facial Emotion in Schizophrenic and Right Brain-Damaged Patients , 1993, The Journal of nervous and mental disease.

[57]  A. Johnston,et al.  Categorizing sex and identity from the biological motion of faces , 2001, Current Biology.

[58]  H. Critchley,et al.  The functional neuroanatomy of social behaviour: changes in cerebral blood flow when people with autistic disorder process facial expressions. , 2000, Brain : a journal of neurology.

[59]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[60]  Christiana M. Leonard,et al.  When's a Smile a Smile? Or how to Detect a Message by Digitizing the Signal , 1991 .

[61]  H. Critchley,et al.  Explicit and implicit neural mechanisms for processing of social information from facial expressions: A functional magnetic resonance imaging study , 2000, Human brain mapping.

[62]  Karl J. Friston,et al.  The physiological basis of attentional modulation in extrastriate visual areas , 1999, Nature Neuroscience.

[63]  Mark S. Cohen,et al.  Changes in cortical activity during mental rotation. A mapping study using functional MRI. , 1996, Brain : a journal of neurology.

[64]  M. Jeannerod,et al.  Mental imaging of motor activity in humans , 1999, Current Opinion in Neurobiology.

[65]  John C. Gore,et al.  Brain activation associated with visual motion studied by functional magnetic resonance imaging in humans , 1994 .

[66]  Michael H. Kutner Applied Linear Statistical Models , 1974 .

[67]  A. Young,et al.  Face processing in psychiatric conditions. , 1992, The British journal of clinical psychology.

[68]  K. Amunts,et al.  Broca's region subserves imagery of motion: A combined cytoarchitectonic and fMRI study , 2000, Human brain mapping.

[69]  Richard S. J. Frackowiak,et al.  Area V5 of the human brain: evidence from a combined study using positron emission tomography and magnetic resonance imaging. , 1993, Cerebral cortex.

[70]  M L Phillips,et al.  Understanding the Symptoms of Schizophrenia Using Visual Scan Paths , 1994, British Journal of Psychiatry.

[71]  Samuel M. McClure,et al.  Predictability Modulates Human Brain Response to Reward , 2001, The Journal of Neuroscience.

[72]  G. Kirouac,et al.  EXPRESSIONS FACIALES EMOTIONNELLES SPONTANEES DYNAMIQUES ET STATIQUES: COMPARAISON D'ETUDES DE JUGEMENT CATEGORIEL ET DIMENSIONNEL , 1995 .

[73]  Karl J. Friston,et al.  A direct demonstration of functional specialization in human visual cortex , 1991, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[74]  Karl J. Friston,et al.  A direct quantitative relationship between the functional properties of human and macaque V5 , 2000, Nature Neuroscience.

[75]  C. Frith,et al.  Reading the mind in cartoons and stories: an fMRI study of ‘theory of mind’ in verbal and nonverbal tasks , 2000, Neuropsychologia.

[76]  G. Pagnoni,et al.  Explicit and Incidental Facial Expression Processing: An fMRI Study , 2001, NeuroImage.

[77]  D. Perrett,et al.  A specific neural substrate for perceiving facial expressions of disgust , 1997, Nature.

[78]  M. Petrides,et al.  Neural correlates of mental transformations of the body-in-space. , 1995, Proceedings of the National Academy of Sciences of the United States of America.

[79]  E. Miller,et al.  The Prefrontal Cortex Complex Neural Properties for Complex Behavior , 1999, Neuron.

[80]  J. Sergent,et al.  Functional neuroanatomy of face and object processing. A positron emission tomography study. , 1992, Brain : a journal of neurology.

[81]  R. Adolphs Neural systems for recognizing emotion , 2002, Current Opinion in Neurobiology.