Emotional expressions in voice and music: Same code, same effect?

Scholars have documented similarities in the way voice and music convey emotions. By using functional magnetic resonance imaging (fMRI) we explored whether these similarities imply overlapping processing substrates. We asked participants to trace changes in either the emotion or pitch of vocalizations and music using a joystick. Compared to music, vocalizations more strongly activated superior and middle temporal cortex, cuneus, and precuneus. However, despite these differences, overlapping rather than differing regions emerged when comparing emotion with pitch tracing for music and vocalizations, respectively. Relative to pitch tracing, emotion tracing activated medial superior frontal and anterior cingulate cortex regardless of stimulus type. Additionally, we observed emotion specific effects in primary and secondary auditory cortex as well as in medial frontal cortex that were comparable for voice and music. Together these results indicate that similar mechanisms support emotional inferences from vocalizations and music and that these mechanisms tap on a general system involved in social cognition. Hum Brain Mapp, 2013. © 2012 Wiley Periodicals, Inc.

[1]  Anders M. Dale,et al.  An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest , 2006, NeuroImage.

[2]  Barbara Tillmann,et al.  The feeling of familiarity of music and odors: the same neural signature? , 2007, Cerebral cortex.

[3]  Marc D. Pell,et al.  Implicit processing of emotional prosody in a foreign versus native language , 2008, Speech Commun..

[4]  Peter Vuust,et al.  Music in minor activates limbic structures: a relationship with dissonance? , 2008, Neuroreport.

[5]  P. Janata.  Music and the self , 2009 .

[6]  Anqi Qiu,et al.  Multi-manifold diffeomorphic metric mapping for aligning cortical hemispheric surfaces , 2010, NeuroImage.

[7]  Werner Strik,et al.  The effect of appraisal level on processing of emotional prosody in meaningless speech , 2008, NeuroImage.

[8]  I. Peretz,et al.  Brain organization for music processing. , 2005, Annual review of psychology.

[9]  M. Masson,et al.  Using confidence intervals in within-subject designs , 1994, Psychonomic bulletin & review.

[10]  J. Mazziotta,et al.  Modulating emotional responses: effects of a neocortical network on the limbic system , 2000, Neuroreport.

[11]  K. Scherer,et al.  Acoustic profiles in vocal emotion expression. , 1996, Journal of personality and social psychology.

[12]  I. Peretz,et al.  Impaired recognition of musical emotions and facial expressions following anteromedial temporal lobe excision , 2011, Cortex.

[13]  R. Patterson,et al.  Task-Dependent Modulation of Medial Geniculate Body Is Behaviorally Relevant for Speech Recognition , 2008, Current Biology.

[14]  D Le Bihan,et al.  Detection of fMRI activation using Cortical Surface Mapping , 2001, Human brain mapping.

[15]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[16]  R. Zatorre,et al.  Human temporal-lobe response to vocal sounds. , 2002, Brain research. Cognitive brain research.

[17]  R. Adolphs,et al.  Amygdala damage impairs emotion recognition from music , 2007, Neuropsychologia.

[18]  Annett Schirmer,et al.  Emotional MMN: Anxiety and heart rate correlate with the ERP signature for auditory change detection , 2010, Clinical Neurophysiology.

[19]  Stefan Zysset,et al.  Gender differences in the activation of inferior frontal cortex during emotional speech perception , 2004, NeuroImage.

[20]  D. Levitin,et al.  Current Advances in the Cognitive Neuroscience of Music , 2009, Annals of the New York Academy of Sciences.

[21]  Mario Dzemidzic,et al.  A cross‐linguistic fMRI study of perception of intonation and emotion in Chinese , 2003, Human brain mapping.

[22]  John G. Csernansky,et al.  Comparing surface-based and volume-based analyses of functional neuroimaging data in patients with schizophrenia , 2008, NeuroImage.

[23]  Jesper Andersson,et al.  Valid conjunction inference with the minimum statistic , 2005, NeuroImage.

[24]  P. Laukka,et al.  Communication of emotions in vocal expression and music performance: different channels, same code? , 2003, Psychological bulletin.

[25]  W. Thompson,et al.  The emergence of music from the Theory of Mind , 2009 .

[26]  W. Thompson,et al.  A Comparison of Acoustic Cues in Music and Speech for Three Dimensions of Affect , 2006 .

[27]  P. Juslin,et al.  Cue Utilization in Communication of Emotion in Music Performance: Relating Performance to Perception Studies of Music Performance , 2022 .

[28]  Michael I. Miller,et al.  Dynamic programming generation of boundaries of local coordinatized submanifolds in the neocortex: application to the planum temporale , 2003, NeuroImage.

[29]  Nikos Makris,et al.  Automatically parcellating the human cerebral cortex. , 2004, Cerebral cortex.

[30]  M. Schönwiesner,et al.  Hemispheric asymmetry for spectral and temporal processing in the human antero‐lateral auditory belt cortex , 2005, The European journal of neuroscience.

[31]  A. Bailey,et al.  Are there theory of mind regions in the brain? A review of the neuroimaging literature , 2009, Human brain mapping.

[32]  Anders M. Dale,et al.  Cortical Surface-Based Analysis I. Segmentation and Surface Reconstruction , 1999, NeuroImage.

[33]  P. Janata.  The Neural Architecture of Music-Evoked Autobiographical Memories , 2009, Cerebral cortex.

[34]  M. Schroeder.  Period histogram and product spectrum: new methods for fundamental-frequency measurement. , 1968, The Journal of the Acoustical Society of America.

[35]  J. Rauschecker,et al.  The role of auditory cortex in the formation of auditory streams , 2007, Hearing Research.

[36]  Aaron C. Koralek,et al.  Two Takes on the Social Brain: A Comparison of Theory of Mind Tasks , 2007, Journal of Cognitive Neuroscience.

[37]  Moo K. Chung,et al.  Cortical thickness analysis in autism with heat kernel smoothing , 2005, NeuroImage.

[38]  H. Ackermann,et al.  Cerebral processing of linguistic and emotional prosody: fMRI studies. , 2006, Progress in brain research.

[39]  Tobias Brosch,et al.  Cross-modal Emotional Attention: Emotional Voices Modulate Early Stages of Visual Processing , 2009, Journal of Cognitive Neuroscience.

[40]  K. Scherer,et al.  The voices of wrath: brain responses to angry prosody in meaningless speech , 2005, Nature Neuroscience.

[41]  N. Bolger,et al.  The neural bases of empathic accuracy , 2009, Proceedings of the National Academy of Sciences.

[42]  R. Zatorre,et al.  Spectral and temporal processing in human auditory cortex. , 2001, Cerebral cortex.

[43]  J. Bharucha,et al.  The minor third communicates sadness in speech, mirroring its use in music. , 2010, Emotion.

[44]  Michael I. Miller,et al.  Estimating linear cortical magnification in human primary visual cortex via dynamic programming , 2006, NeuroImage.

[45]  M. Erb,et al.  Dynamic Brain Activation during Processing of Emotional Intonation: Influence of Acoustic Parameters, Emotional Valence, and Sex , 2002, NeuroImage.

[46]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[47]  M. Peelen,et al.  Supramodal Representations of Perceived Emotions in the Human Brain , 2010, The Journal of Neuroscience.

[48]  Klaus R. Scherer,et al.  Vocal communication of emotion: A review of research paradigms , 2003, Speech Commun..

[49]  A. Young,et al.  Neural responses to facial and vocal expressions of fear and disgust , 1998, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[50]  J. Pittam.  Voice in Social Interaction: An Interdisciplinary Approach , 1994 .

[51]  Lok Yap,et al.  How does the brain process music? , 2008, Clinical medicine.

[52]  Nancy Kanwisher,et al.  Divide and conquer: A defense of functional localizers , 2006, NeuroImage.

[53]  R. Zatorre,et al.  Voice-selective areas in human auditory cortex , 2000, Nature.

[54]  I. Peretz,et al.  Universal Recognition of Three Basic Emotions in Music , 2009, Current Biology.

[55]  C. Liégeois-Chauvel,et al.  Brain regions involved in the recognition of happiness and sadness in music , 2005, Neuroreport.

[56]  Erich Schröger,et al.  From Air Oscillations to Music and Speech: Functional Magnetic Resonance Imaging Evidence for Fine-Tuned Neural Networks in Audition , 2006, The Journal of Neuroscience.

[57]  K. Scherer.  Vocal affect expression: a review and a model for future research. , 1986, Psychological bulletin.

[58]  M. Erb,et al.  Effects of prosodic emotional intensity on activation of associative auditory cortex , 2006, Neuroreport.

[59]  H. Spencer.  The Origin and Function of Music , 2015 .

[60]  D. Weinberger,et al.  Neocortical modulation of the amygdala response to fearful stimuli , 2003, Biological Psychiatry.

[61]  Steven C. R. Williams,et al.  A functional MRI study of happy and sad affective states induced by classical music , 2007, Human brain mapping.

[62]  Anqi Qiu,et al.  Quantitative evaluation of LDDMM, FreeSurfer, and CARET for cortical surface mapping , 2010, NeuroImage.

[63]  Richard S. J. Frackowiak,et al.  Other minds in the brain: a functional imaging study of “theory of mind” in story comprehension , 1995, Cognition.

[64]  J. Rousseau,et al.  The first and second discourses together with the replies to critics and Essay on the origin of languages , 1986 .

[65]  Pascal Belin,et al.  Amygdala responses to nonlinguistic emotional vocalizations , 2007, NeuroImage.

[66]  Daniel C. Javitt,et al.  “It's Not What You Say, But How You Say it”: A Reciprocal Temporo-frontal Network for Affective Prosody , 2009, Front. Hum. Neurosci..

[67]  R. Adolphs,et al.  Emotional responses to unpleasant music correlates with damage to the parahippocampal cortex. , 2006, Brain : a journal of neurology.

[68]  H. Critchley,et al.  Explicit and implicit neural mechanisms for processing of social information from facial expressions: A functional magnetic resonance imaging study , 2000, Human brain mapping.

[69]  B. Croisile,et al.  Progressive amusia and aprosody. , 1992, Archives of neurology.

[70]  N. Kraus,et al.  Musical experience and neural efficiency – effects of training on subcortical processing of vocal expressions of emotion , 2009, The European journal of neuroscience.

[71]  K Mori,et al.  Task‐dependent laterality for cue decoding during spoken language processing , 1998, Neuroreport.

[72]  André Brechmann,et al.  Auditory stream segregation relying on timbre involves left auditory cortex , 2004, Neuroreport.

[73]  Friedemann Pulvermüller,et al.  Determinants of dominance: Is language laterality explained by physical or linguistic features of speech? , 2005, NeuroImage.

[74]  J. Thayer,et al.  Music and Autonomic Nervous System (Dys)function. , 2010, Music perception.

[75]  K. Scherer,et al.  Emotions evoked by the sound of music: characterization, classification, and measurement. , 2008, Emotion.

[76]  Annett Schirmer,et al.  Perceiving verbal and vocal emotions in a second language , 2011, Cognition & emotion.

[77]  I. Peretz,et al.  Impaired recognition of scary music following unilateral temporal lobe excision. , 2005, Brain : a journal of neurology.

[78]  Angela D. Friederici,et al.  When vocal processing gets emotional: On the role of social orientation in relevance detection by the human amygdala , 2008, NeuroImage.

[79]  I. Peretz,et al.  Music and emotion: perceptual determinants, immediacy, and isolation after brain damage , 1998, Cognition.

[80]  Masatoshi Itoh,et al.  Recruitment of fusiform face area associated with listening to degraded speech sounds in auditory–visual speech perception: a PET study , 2005, Neuroscience Letters.

[81]  P. Janata.  Brain Networks That Track Musical Structure , 2005, Annals of the New York Academy of Sciences.

[82]  Jean-Jacques Rousseau,et al.  Essay on the Origin of Languages and Writings Related to Music , 1998 .

[83]  Karl J. Friston,et al.  Generalisability, Random Effects & Population Inference , 1998, NeuroImage.

[84]  E. T. Possing,et al.  Human temporal lobe activation by speech and nonspeech sounds. , 2000, Cerebral cortex.

[85]  S. Kotz,et al.  Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing , 2006, Trends in Cognitive Sciences.

[86]  Michael I. Miller,et al.  Smooth functional and structural maps on the neocortex via orthonormal bases of the Laplace-Beltrami operator , 2006, IEEE Transactions on Medical Imaging.

[87]  Karl J. Friston,et al.  A unified statistical approach for determining significant signals in images of cerebral activation , 1996, Human brain mapping.

[88]  Gilles Pourtois,et al.  Emotion and attention interactions in social cognition: Brain regions involved in processing anger prosody , 2005, NeuroImage.

[89]  R. Zatorre,et al.  Structure and function of auditory cortex: music and speech , 2002, Trends in Cognitive Sciences.

[90]  R. Turner,et al.  Event-Related fMRI: Characterizing Differential Responses , 1998, NeuroImage.

[91]  C. Frith,et al.  Reading the mind in cartoons and stories: an fMRI study of ‘theory of mind’ in verbal and nonverbal tasks , 2000, Neuropsychologia.