Cerebral representation of non-verbal emotional perception: fMRI reveals audiovisual integration area between voice- and face-sensitive regions in the superior temporal sulcus

Successful social interaction relies on multimodal integration of non-verbal emotional signals. The neural correlates of this function, along with those underlying the processing of human faces and voices, have been linked to the superior temporal sulcus (STS) in previous neuroimaging studies. Yet, recently it has been demonstrated that this structure consists of several anatomically defined sections, including a trunk section as well as two separate terminal branches, and exhibits a pronounced spatial variability across subjects. Using functional magnetic resonance imaging (fMRI), we demonstrated that the neural representations of the audiovisual integration of non-verbal emotional signals, voice sensitivity and face sensitivity are located in different parts of the STS with maximum voice sensitivity in the trunk section and maximum face sensitivity in the posterior terminal ascending branch. The audiovisual integration area for emotional signals is located at the bifurcation of the STS at an overlap of voice- and face-sensitive regions. In summary, our findings evidence a functional subdivision of the STS into modules subserving the processing of different aspects of social communication, here exemplified in human voices and faces and audiovisual integration of emotional signals from these sources and suggest a possible interaction of the underlying voice- and face-sensitive neuronal populations during the formation of the audiovisual emotional percept.

[1]  Jean-Francois Mangin,et al.  Sulcal pattern and morphology of the superior temporal sulcus , 2004, NeuroImage.

[2]  J. Decety,et al.  Top down effect of strategy on the perception of human biological motion: a pet investigation. , 1998, Cognitive neuropsychology.

[3]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[4]  B. Chapman,et al.  Incremental Validity of a Measure of Emotional Intelligence , 2005, Journal of personality assessment.

[5]  Michael Erb,et al.  Audiovisual integration of emotional signals in voice and face: An event-related fMRI study , 2007, NeuroImage.

[6]  Anne-Lise Giraud,et al.  Distinct functional substrates along the right superior temporal sulcus for the processing of voices , 2004, NeuroImage.

[7]  B. Rossion,et al.  The time‐course of intermodal binding between seeing and hearing affective information , 2000, Neuroreport.

[8]  J. Malouff,et al.  Development and validation of a measure of emotional intelligence. , 1998 .

[9]  R. Veit,et al.  Differential cerebral activation during observation of expressive gestures and motor acts , 2006, Neuropsychologia.

[10]  Gregory McCarthy,et al.  Polysensory interactions along lateral temporal regions evoked by audiovisual speech. , 2003, Cerebral cortex.

[11]  Leslie G. Ungerleider,et al.  Repetition suppression of faces is modulated by emotion. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[12]  E. Austin,et al.  Factor structure and validity of a trait emotional intelligence measure , 2003 .

[13]  P Sterzer,et al.  Contributions of sensory input, auditory search and verbal comprehension to cortical activity during speech processing. , 2004, Cerebral cortex.

[14]  Russell A. Epstein,et al.  The Parahippocampal Place Area Recognition, Navigation, or Encoding? , 1999, Neuron.

[15]  T. Allison,et al.  Social perception from visual cues: role of the STS region , 2000, Trends in Cognitive Sciences.

[16]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[17]  Bruno Rossion,et al.  Understanding the functional neuroanatomy of acquired prosopagnosia , 2007, NeuroImage.

[18]  Pascal Belin,et al.  Is voice processing species-specific in human auditory cortex? An fMRI study , 2004, NeuroImage.

[19]  M. Bradley,et al.  Measuring emotion: the Self-Assessment Manikin and the Semantic Differential. , 1994, Journal of behavior therapy and experimental psychiatry.

[20]  S. Geisser,et al.  An Extension of Box's Results on the Use of the $F$ Distribution in Multivariate Analysis , 1958 .

[21]  J. Vroomen,et al.  The perception of emotions by ear and by eye , 2000 .

[22]  Gilles Pourtois,et al.  Investigating audiovisual integration of emotional signals in the human brain. , 2006, Progress in brain research.

[23]  D. Pandya,et al.  Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey , 1978, Brain Research.

[24]  R. Goebel,et al.  Integration of Letters and Speech Sounds in the Human Brain , 2004, Neuron.

[25]  Aaron C. Koralek,et al.  Two Takes on the Social Brain: A Comparison of Theory of Mind Tasks , 2007, Journal of Cognitive Neuroscience.

[26]  Turhan Canli,et al.  Individual differences in emotion processing , 2004, Current Opinion in Neurobiology.

[27]  John M. Malouff,et al.  Emotional Intelligence and Interpersonal Relations , 2001, The Journal of social psychology.

[28]  G. Pourtois,et al.  Perception of Facial Expressions and Voices and of their Combination in the Human Brain , 2005, Cortex.

[29]  N. Logothetis,et al.  A voice region in the monkey brain , 2008, Nature Neuroscience.

[30]  Eleanor A. Maguire,et al.  Spontaneous mentalizing during an interactive real world task: An fMRI study , 2006, Neuropsychologia.

[31]  N. Kanwisher,et al.  The Fusiform Face Area: A Module in Human Extrastriate Cortex Specialized for Face Perception , 1997, The Journal of Neuroscience.

[32]  D. Massaro,et al.  Perceiving affect from the voice and the face , 1996, Psychonomic bulletin & review.

[33]  Doris Y. Tsao,et al.  A Cortical Region Consisting Entirely of Face-Selective Cells , 2006, Science.

[34]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[35]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[36]  J. Mayer,et al.  Convergent, Discriminant, and Incremental Validity of Competing Measures of Emotional Intelligence , 2003, Personality & social psychology bulletin.

[37]  Chris I. Baker,et al.  Integration of Visual and Auditory Information by Superior Temporal Sulcus Neurons Responsive to the Sight of Actions , 2005, Journal of Cognitive Neuroscience.

[38]  R. Zatorre,et al.  Voice-selective areas in human auditory cortex , 2000, Nature.