The neural mechanism of imagining facial affective expression

To react appropriately in social relationships, we have a tendency to simulate how others think of us through mental imagery. In particular, simulating other people's facial affective expressions through imagery in social situations enables us to enact vivid affective responses, which may be inducible from other people's affective responses that are predicted as results of our mental imagery of future behaviors. Therefore, this ability is an important cognitive feature of diverse advanced social cognition in humans. We used functional magnetic imaging to examine brain activation during the imagery of emotional facial expressions as compared to neutral facial expressions. Twenty-one right-handed subjects participated in this study. We observed the activation of the amygdala during the imagining of emotional facial affect versus the imagining of neutral facial affects. In addition, we also observed the activation of several areas of the brain, including the dorsolateral prefrontal cortex, ventral premotor cortex, superior temporal sulcus, parahippocampal gyrus, lingual gyrus, and the midbrain. Our results suggest that the areas of the brain known to be involved in the actual perception of affective facial expressions are also implicated in the imagery of affective facial expressions. In particular, given that the processing of information concerning the facial patterning of different emotions and the enactment of behavioral responses, such as autonomic arousal, are central components of the imagery of emotional facial expressions, we postulate the central role of the amygdala in the imagery of emotional facial expressions.

[1]  Michael Davis,et al.  Neurobiology of fear responses: the role of the amygdala. , 1997, The Journal of neuropsychiatry and clinical neurosciences.

[2]  U. Dimberg,et al.  Unconscious Facial Reactions to Emotional Facial Expressions , 2000, Psychological science.

[3]  G. Rizzolatti,et al.  Action recognition in the premotor cortex. , 1996, Brain : a journal of neurology.

[4]  A. Liberman,et al.  On the relation of speech to language , 2000, Trends in Cognitive Sciences.

[5]  Michael Erb,et al.  Why are smiles contagious? An fMRI study of the interaction between perception of facial affect and facial movements , 2003, Psychiatry Research: Neuroimaging.

[6]  A. Benton,et al.  Frontal Lobe Function and Dysfunction , 1991 .

[7]  K. Scherer,et al.  Handbook of affective sciences. , 2003 .

[8]  Leslie G. Ungerleider,et al.  Visual Imagery of Famous Faces: Effects of Memory and Attention Revealed by fMRI , 2002, NeuroImage.

[9]  K Ugurbil,et al.  Human primary visual cortex and lateral geniculate nucleus activation during visual imagery , 1998, Neuroreport.

[10]  E. Naito,et al.  Enhanced neural activity in response to dynamic facial expressions of emotion: an fMRI study. , 2004, Brain research. Cognitive brain research.

[11]  S. Kosslyn,et al.  Visual Mental Imagery Activates Topographically Organized Visual Cortex: PET Investigations , 1993, Journal of Cognitive Neuroscience.

[12]  B. Gulyás,et al.  Visual memory, visual imagery, and visual recognition of large field patterns by the human brain: functional anatomy by positron emission tomography. , 1995, Cerebral cortex.

[13]  Christian Scheiber,et al.  Neural substrates of animal mental imagery: calcarine sulcus and dorsal pathway involvement — an fMRI study , 2002, Brain Research.

[14]  N. Sadato,et al.  Age‐related differences in the medial temporal lobe responses to emotional faces as revealed by fMRI , 2002, Hippocampus.

[15]  U. Habel,et al.  Gender differences in regional cerebral activity during sadness , 2000, Human brain mapping.

[16]  M. Hajós,et al.  Evidence for a role of GABA interneurones in the cortical modulation of midbrain 5-hydroxytryptamine neurones , 2001, Neuroscience.

[17]  Patrik Vuilleumier,et al.  Emotional facial expressions capture attention , 2001, Neurology.

[18]  A. Sirigu,et al.  Differential amygdala responses to winning and losing: a functional magnetic resonance imaging study in humans , 2000, The European journal of neuroscience.

[19]  U. Dimberg,et al.  Rapid facial reactions to emotional facial expressions. , 1998, Scandinavian journal of psychology.

[20]  S. Kosslyn,et al.  Neural foundations of imagery , 2001, Nature Reviews Neuroscience.

[21]  R. Zatorre,et al.  Odor-induced changes in taste perception , 2004, Experimental Brain Research.

[22]  A. Halpern,et al.  Cerebral Substrates of Musical Imagery , 2001, Annals of the New York Academy of Sciences.

[23]  E. Bullmore,et al.  Investigation of facial recognition memory and happy and sad facial expression perception: an fMRI study , 1998, Psychiatry Research: Neuroimaging.

[24]  J. Mazziotta,et al.  Neural mechanisms of empathy in humans: A relay from neural systems for imitation to limbic areas , 2003, Proceedings of the National Academy of Sciences of the United States of America.

[25]  K. Heilman,et al.  Emotional facial imagery, perception, and expression in Parkinson's disease , 1995, Neurology.

[26]  R. Gur,et al.  The effects of right and left hemiparkinsonism on prosody , 1989, Brain and Language.

[27]  G. Horstmann.  What do facial expressions convey: feeling states, behavioral intentions, or action requests? , 2003, Emotion.

[28]  Karl J. Friston,et al.  Where bottom-up meets top-down: neuronal interactions during perception and imagery. , 2004, Cerebral cortex.

[29]  Katsuki Nakamura,et al.  Visual response properties of neurons in the parahippocampal cortex of monkeys. , 2003, Journal of neurophysiology.

[30]  C. V. Witvliet,et al.  Psychophysiological responses as indices of affective dimensions. , 1995, Psychophysiology.

[31]  Seung-Schik Yoo,et al.  Neural substrates of tactile imagery: a functional MRI study , 2003, Neuroreport.

[32]  R. E. Gur,et al.  Effects of Emotional Discrimination Tasks on Cerebral Blood Flow: Regional Activation and Its Relation to Performance , 1994, Brain and Cognition.

[33]  Scott T Grafton,et al.  Functional imaging of face and hand imitation: towards a motor theory of empathy , 2004, NeuroImage.

[34]  M. Pell.  On the Receptive Prosodic Loss in Parkinson's Disease , 1996, Cortex.

[35]  T. Allison,et al.  Temporal Cortex Activation in Humans Viewing Eye and Mouth Movements , 1998, The Journal of Neuroscience.

[36]  R. Maddock,et al.  Time course of the subjective emotional response to aversive pictures: relevance to fMRI studies , 2001, Psychiatry Research: Neuroimaging.

[37]  W. Grodd,et al.  Differential amygdala activation in schizophrenia during sadness , 1998, Schizophrenia Research.

[38]  A. Damasio,et al.  The return of Phineas Gage: clues about the brain from the skull of a famous patient. , 1994, Science.

[39]  David C. Alsop,et al.  Brain Activation during Facial Emotion Processing , 2002, NeuroImage.

[40]  G. Rizzolatti,et al.  Premotor cortex and the recognition of motor actions. , 1996, Brain research. Cognitive brain research.

[41]  R. Dolan,et al.  The role of the prefrontal cortex in higher cognitive functions. , 1996, Brain research. Cognitive brain research.

[42]  J. Decety,et al.  Brain Regions Involved in the Perception of Gaze: A PET Study , 1998, NeuroImage.

[43]  M. Bradley,et al.  Emotion, Motivation, and Anxiety: Brain Mechanisms and Psychophysiology the Motivational Organization of Emotion Patterns of Human Emotion Emotion and Perception the Psychophysiology of Picture Processing Neural Imaging: Motivation in the Visual Cortex Motivational Circuits in the Brain , 2022 .

[44]  R. Wong,et al.  Secondary Activation of a Cation Conductance Is Responsible for NMDA Toxicity in Acutely Isolated Hippocampal Neurons , 1997, The Journal of Neuroscience.

[45]  T. Benke,et al.  A Study of Emotional Processing in Parkinson's Disease , 1998, Brain and Cognition.

[46]  U. Eysel,et al.  Neural structures associated with recognition of facial expressions of basic emotions , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[47]  June P Tangney,et al.  Proneness to shame, proneness to guilt, and psychopathology. , 1992, Journal of Abnormal Psychology.

[48]  A M Liberman,et al.  Perception of the speech code. , 1967, Psychological review.

[49]  M. Petrides,et al.  Functional neuroimaging of odor imagery , 2005, NeuroImage.

[50]  M. Denis,et al.  Cortical anatomy of mental imagery of concrete nouns based on their dictionary definition , 1998, Neuroreport.

[51]  G. McCarthy,et al.  Dissociable prefrontal brain systems for attention and emotion , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[52]  R. Dolan,et al.  Neural Activation during Covert Processing of Positive Emotional Facial Expressions , 1996, NeuroImage.

[53]  A. Young,et al.  Neural responses to facial and vocal expressions of fear and disgust , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[54]  L. Nadel,et al.  Cognitive neuroscience of emotion , 2000 .

[55]  Hans Forssberg,et al.  Increased Brain Activity in Frontal and Parietal Cortex Underlies the Development of Visuospatial Working Memory Capacity during Childhood , 2002, Journal of Cognitive Neuroscience.

[56]  P. Goldman-Rakic,et al.  The physiological approach: functional architecture of working memory and disordered cognition in schizophrenia , 1999, Biological Psychiatry.

[57]  J. O'Doherty,et al.  Automatic and intentional brain responses during evaluation of trustworthiness of faces , 2002, Nature Neuroscience.

[58]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[59]  S. Preston,et al.  Empathy: Its ultimate and proximate bases. , 2001, The Behavioral and brain sciences.

[60]  G. Pagnoni,et al.  Explicit and Incidental Facial Expression Processing: An fMRI Study , 2001, NeuroImage.

[61]  Clinton D. Kilts,et al.  Dissociable Neural Pathways Are Involved in the Recognition of Emotion in Static and Dynamic Facial Expressions , 2003, NeuroImage.

[62]  R. Kahn,et al.  Lateralization of amygdala activation: a systematic review of functional neuroimaging studies , 2004, Brain Research Reviews.

[63]  J. Talairach,et al.  Co-Planar Stereotaxic Atlas of the Human Brain: 3-Dimensional Proportional System: An Approach to Cerebral Imaging , 1988 .

[64]  M. Annett.  A classification of hand preference by association analysis. , 1970, British journal of psychology.

[65]  N. Kanwisher.  Domain specificity in face perception , 2000, Nature Neuroscience.

[66]  W. Singer,et al.  The constructive nature of vision: direct evidence from functional magnetic resonance imaging studies of apparent motion and motion imagery , 1998, The European journal of neuroscience.

[67]  I. Daum,et al.  Impaired Perception of Vocal Emotions in Parkinson's Disease: Influence of Speech Time Processing and Executive Functioning , 2001, Brain and Cognition.

[68]  Peter Erhard,et al.  A common neural basis for receptive and expressive communication of pleasant facial affect , 2005, NeuroImage.

[69]  A. Liberman,et al.  The motor theory of speech perception revised , 1985, Cognition.

[70]  K. Zilles,et al.  Subcortical correlates of craving in recently abstinent alcoholic patients. , 2001, The American journal of psychiatry.

[71]  S. Kosslyn,et al.  Mental Imagery of High- and Low-Resolution Gratings Activates Area 17 , 2001, NeuroImage.

[72]  N. Kanwisher,et al.  Mental Imagery of Faces and Places Activates Corresponding Stimulus-Specific Brain Regions , 2000, Journal of Cognitive Neuroscience.

[73]  P. Morgane,et al.  A review of systems and networks of the limbic forebrain/limbic midbrain , 2005, Progress in Neurobiology.

[74]  C. Patrick,et al.  Affective imagery and the startle response: probing mechanisms of modulation during pleasant scenes, personal experiences, and discrete negative emotions. , 2002, Psychophysiology.

[75]  Robin I. M. Dunbar,et al.  Primate cognition: from ‘what now?’ to ‘what if?’ , 2003, Trends in Cognitive Sciences.

[76]  P F Renshaw,et al.  Functional magnetic resonance imaging of facial affect recognition in children and adolescents. , 1999, Journal of the American Academy of Child and Adolescent Psychiatry.

[77]  R. Bajcsy,et al.  A method for obtaining 3-dimensional facial expressions and its standardization for use in neurocognitive studies , 2002, Journal of Neuroscience Methods.

[78]  J. Borod,et al.  Parameters of emotional processing in neuropsychiatric disorders: conceptual issues and a battery of tests. , 1990, Journal of communication disorders.

[79]  D. Perrett,et al.  Dissociable neural responses to facial expressions of sadness and anger. , 1999, Brain : a journal of neurology.

[80]  G. Rizzolatti,et al.  Motor and cognitive functions of the ventral premotor cortex , 2002, Current Opinion in Neurobiology.

[81]  P. Ekman,et al.  Autonomic nervous system activity distinguishes among emotions. , 1983, Science.

[82]  Francesco Fera,et al.  Dopamine Modulates the Response of the Human Amygdala: A Study in Parkinson's Disease , 2002, The Journal of Neuroscience.

[83]  Deborah J. Hellawell,et al.  Facial expression processing after amygdalotomy , 1996, Neuropsychologia.

[84]  Leslie G. Ungerleider,et al.  Distributed Neural Systems for the Generation of Visual Images , 2000, Neuron.

[85]  R. Adolphs,et al.  Impaired Recognition of Social Emotions following Amygdala Damage , 2002, Journal of Cognitive Neuroscience.

[86]  Joseph E LeDoux.  Emotion Circuits in the Brain , 2000 .

[87]  D. Ingvar,et al.  Distribution of cerebral blood flow in the dominant hemisphere during motor ideation and motor performance , 1977, Annals of neurology.

[88]  G. Weber-Luxenburger,et al.  Psychic trauma causing grossly reduced brain metabolism and cognitive deterioration , 1998, Neuropsychologia.

[89]  U. Schroeder,et al.  Functional neuroanatomy of perceiving surprised faces , 2004, Human brain mapping.

[90]  Noriaki Hattori,et al.  Functional Imaging of Gustatory Perception and Imagery: Btop-downq Processing of Gustatory Signals , 2004 .

[91]  R. Nebes,et al.  Patterns of Hand Preference in a Student Population , 1975, Cortex.

[92]  Ralph Adolphs,et al.  Processing of the Arousal of Subliminal and Supraliminal Emotional Stimuli by the Human Amygdala , 2003, The Journal of Neuroscience.

[93]  S. Rauch,et al.  Response and Habituation of the Human Amygdala during Visual Processing of Facial Expression , 1996, Neuron.

[94]  John D E Gabrieli,et al.  Amygdala Response to Happy Faces as a Function of Extraversion , 2002, Science.

[95]  M. Lowe,et al.  Human amygdala activation detected with echo-planar functional magnetic resonance imaging. , 1996, Neuroreport.

[96]  Karl J. Friston,et al.  A neuromodulatory role for the human amygdala in processing emotional facial expressions. , 1998, Brain : a journal of neurology.

[97]  Hermann Ackermann,et al.  Emotional processing following cortical and subcortical brain damage: contribution of the fronto-striatal circuitry. , 1998, Behavioural neurology.

[98]  D. Perrett,et al.  A differential neural response in the human amygdala to fearful and happy facial expressions , 1996, Nature.

[99]  P. Ekman,et al.  Voluntary facial action generates emotion-specific autonomic nervous system activity. , 1990, Psychophysiology.

[100]  M. Farah,et al.  A functional MRI study of mental image generation , 1997, Neuropsychologia.

[101]  J. Haxby,et al.  Distinct representations of eye gaze and identity in the distributed human neural system for face perception , 2000, Nature Neuroscience.