Neural architecture underlying person perception from in-group and out-group voices

ABSTRACT In spoken language, verbal cues (what we say) and vocal cues (how we say it) contribute to person perception, the process for interpreting information and making inferences about other people. When someone has an accent, forming impressions from the speaker's voice may be influenced by social categorization processes (i.e., activating stereotypical traits of members of a perceived ‘out‐group’) and by processes which differentiate the speaker based on their individual attributes (e.g., registering the vocal confidence level of the speaker in order to make a trust decision). The neural systems for using vocal cues that refer to the speaker's identity and to qualities of their vocal expression to generate inferences about others are not known. Here, we used functional magnetic resonance imaging (fMRI) to investigate how speaker categorization influences brain activity as Canadian‐English listeners judged whether they believe statements produced by in‐group (native) and out‐group (regional, foreign) speakers. Each statement was expressed in a confident, doubtful, and neutral tone of voice. In‐group speakers were perceived as more believable than speakers with out‐group accents overall, confirming social categorization of speakers based on their accent. Superior parietal and middle temporal regions were uniquely activated when listening to out‐group compared to in‐group speakers suggesting that they may be involved in extracting the attributes of speaker believability from the lower‐level acoustic variations. Basal ganglia, left cuneus and right fusiform gyrus were activated by confident expressions produced by out‐group speakers. These regions appear to participate in abstracting more ambiguous believability attributes from accented speakers (where a conflict arises between the tendency to disbelieve an out‐group speaker and the tendency to believe a confident voice). For out‐group speakers, stronger impressions of believability selectively modulated activity in the bilateral superior and middle temporal regions. Moreover, the right superior temporal gyrus, a region that was associated with perceived speaker confidence, was found to be functionally connected to the left lingual gyrus and right middle temporal gyrus when out‐group speakers were judged as more believable. These findings suggest that identity‐related voice characteristics and associated biases may influence underlying neural activities for making social attributions about out‐group speakers, affecting decisions about believability and trust. Specifically, inferences about out‐group speakers seem to be mediated to a greater extent by stimulus‐related features (i.e., vocal confidence cues) than for in‐group speakers. Our approach highlights how the voice can be studied to advance models of person perception. HIGHLIGHTSNeural activations of social inference from “out‐group” voices were examined with fMRI.Basal ganglia, left cuneus and right fusiform gyrus were enhanced by confident voices of a speaker with an accent.Connectivity between the right STG and the left lingual gyrus and right MTG increased when judging believability of an out‐group speaker.Listener attitude and intelligibility perception modulated the brain network associated with speaker believability.

[1]  Agnieszka Sorokowska,et al.  Voice-based assessments of trustworthiness, competence, and warmth in blind and sighted adults , 2016, Psychonomic bulletin & review.

[2]  Feng Sheng,et al.  Manipulations of cognitive strategies and intergroup relationships reduce the racial bias in empathic neural responses , 2012, NeuroImage.

[3]  M. Pell,et al.  Cultural immersion alters emotion perception: Neurophysiological evidence from Chinese immigrants to Canada , 2016, Social neuroscience.

[4]  Stephen M. Smith,et al.  General multilevel linear modeling for group analysis in FMRI , 2003, NeuroImage.

[5]  A. Todorov,et al.  How Do You Say ‘Hello’? Personality Impressions from Brief Novel Voices , 2014, PloS one.

[6]  M. Pell,et al.  Neural systems for evaluating speaker (Un)believability , 2017, Human brain mapping.

[7]  Emmanuel Dupoux,et al.  Perceptual adjustment to highly compressed speech: effects of talker and rate changes. , 1997, Journal of experimental psychology. Human perception and performance.

[8]  R. Janney,et al.  Toward a pragmatics of emotive communication , 1994 .

[9]  Ariel Woodbury,et al.  How do you feel? , 2014, Nursing.

[10]  Pascal Belin,et al.  Right temporal TMS impairs voice detection , 2011, Current Biology.

[11]  Meghan Sumner The social weight of spoken words , 2015, Trends in Cognitive Sciences.

[12]  Annett Schirmer,et al.  Is the voice an auditory face? An ALE meta-analysis comparing vocal and facial emotion processing , 2017, Social cognitive and affective neuroscience.

[13]  Jason P. Mitchell,et al.  Feeling-of-knowing in episodic memory: an event-related fMRI study , 2003, NeuroImage.

[14]  T. D. Hanley,et al.  Effects of phonological speech foreignness upon three dimensions of attitude of selected American listeners , 1974 .

[15]  A. Greenwald,et al.  Measuring individual differences in implicit cognition: the implicit association test. , 1998, Journal of personality and social psychology.

[16]  M. Pell,et al.  On how the brain decodes vocal cues about speaker confidence , 2015, Cortex.

[17]  J. Mattingley,et al.  Brain regions with mirror properties: A meta-analysis of 125 human fMRI studies , 2012, Neuroscience & Biobehavioral Reviews.

[18]  Frank Van Overwalle,et al.  Involvement of the mentalizing network in social and non-social high construal. , 2014, Social cognitive and affective neuroscience.

[19]  Ladan Ghazi-Saidi,et al.  How native-like can you possibly get: fMRI evidence for processing accent , 2015, Front. Hum. Neurosci..

[20]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[21]  R. Passingham,et al.  Action observation and acquired motor skills: an FMRI study with expert dancers. , 2005, Cerebral cortex.

[22]  R. Sebastian,et al.  The effects of speech style and social class background on social judgements of speakers , 1980 .

[23]  Shihui Han,et al.  A Culture–Behavior–Brain Loop Model of Human Development , 2015, Trends in Cognitive Sciences.

[24]  A. Friederici,et al.  Lateralization of auditory language functions: A dynamic dual pathway model , 2004, Brain and Language.

[25]  Shihui Han Neurocognitive Basis of Racial Ingroup Bias in Empathy , 2018, Trends in Cognitive Sciences.

[26]  Mark W. Woolrich,et al.  Multilevel linear modelling for FMRI group analysis using Bayesian inference , 2004, NeuroImage.

[27]  F. Overwalle Social cognition and the brain: a meta-analysis. , 2009 .

[28]  Xiaoming Jiang,et al.  Effects of contextual relevance on pragmatic inference during conversation: An fMRI study , 2017, Brain and Language.

[29]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[30]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[31]  Guangyu Chen,et al.  Negative Functional Connectivity and Its Dependence on the Shortest Path Length of Positive Network in the Resting-State Human Brain , 2011, Brain Connect..

[32]  Werner Strik,et al.  The effect of appraisal level on processing of emotional prosody in meaningless speech , 2008, NeuroImage.

[33]  M. Pell,et al.  Neural correlates of inferring speaker sincerity from white lies: An event-related potential source localization study , 2014, Brain Research.

[34]  Tyler K. Perrachione,et al.  Asymmetric cultural effects on perceptual expertise underlie an own-race bias for voices , 2010, Cognition.

[35]  John C. Mazziotta,et al.  A Probabilistic Atlas and Reference System for the Human Brain , 2001 .

[36]  Peter Hagoort,et al.  UvA-DARE (Digital Academic Repository) Unification of speaker and meaning in language comprehension: an fMRI study , 2022 .

[37]  K. Stevens,et al.  Linguistic experience alters phonetic perception in infants by 6 months of age. , 1992, Science.

[38]  D. Sammler,et al.  Neural bases of social communicative intentions in speech , 2018, Social cognitive and affective neuroscience.

[39]  Xiaoming Jiang,et al.  The sound of confidence and doubt , 2017, Speech Commun..

[40]  Rachel M. Theodore,et al.  Language exposure facilitates talker learning prior to language comprehension, even in adults , 2015, Cognition.

[41]  D. Amodio The neuroscience of prejudice and stereotyping , 2014, Nature Reviews Neuroscience.

[42]  Friedemann Pulvermüller,et al.  Brain basis of communicative actions in language , 2016, NeuroImage.

[43]  Sven Joubert,et al.  Preserved visual recognition memory in an amnesic patient with hippocampal lesions , 2005, Hippocampus.

[44]  Alan C. Evans,et al.  Event-related fMRI of the auditory cortex. , 1998, NeuroImage.

[45]  Hermann Ackermann,et al.  The contribution of the insula to motor aspects of speech production: A review and a hypothesis , 2004, Brain and Language.

[46]  Yi Rao,et al.  Oxytocin receptor gene and racial ingroup bias in empathy-related brain activity , 2015, NeuroImage.

[47]  D. Pisoni,et al.  Talker-specific learning in speech perception , 1998, Perception & psychophysics.

[48]  K. Scherer,et al.  The voices of wrath: brain responses to angry prosody in meaningless speech , 2005, Nature Neuroscience.

[49]  Stephen M. Smith,et al.  Temporal Autocorrelation in Univariate Linear Modeling of FMRI Data , 2001, NeuroImage.

[50]  J. Rauschecker,et al.  Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing , 2009, Nature Neuroscience.

[51]  Dominic S. Fareri,et al.  Race and reputation: perceived racial group trustworthiness influences the neural correlates of trust decisions , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[52]  Frank Van Overwalle,et al.  Social cognition and the cerebellum: A meta-analysis of over 350 fMRI studies , 2014, NeuroImage.

[53]  Shiri Lev-Ari,et al.  Comprehending non-native speakers: theory and evidence for adjustment in manner of processing , 2014, Front. Psychol..

[54]  S. Campanella,et al.  Integrating face and voice in person perception , 2007, Trends in Cognitive Sciences.

[55]  Xiaoying Wang,et al.  Do You Feel My Pain? Racial Group Membership Modulates Empathic Neural Responses , 2009, The Journal of Neuroscience.

[56]  Wei Zhang,et al.  Selective aberrant functional connectivity of resting state networks in social anxiety disorder , 2010, NeuroImage.

[57]  Carrie L. Masten,et al.  An fMRI investigation of empathy for ‘social pain’ and subsequent prosocial behavior , 2011, NeuroImage.

[58]  Pascal Belin,et al.  Implicitly perceived vocal attractiveness modulates prefrontal cortex activity. , 2012, Cerebral cortex.

[59]  Peter Hagoort,et al.  The Neural Integration of Speaker and Message , 2008, Journal of Cognitive Neuroscience.

[60]  Alan C. Evans,et al.  A general statistical analysis for fMRI data , 2000, NeuroImage.

[61]  B. Keysar,et al.  Why don't we believe non-native speakers? The influence of accent on credibility , 2010 .

[62]  P. Hagoort,et al.  Empathy matters: ERP evidence for inter-individual differences in social language processing , 2010, Social cognitive and affective neuroscience.

[63]  A. Friederici Towards a neural basis of auditory sentence processing , 2002, Trends in Cognitive Sciences.

[64]  S. Kotz,et al.  Beyond the right hemisphere: brain mechanisms mediating vocal emotional processing , 2006, Trends in Cognitive Sciences.

[65]  Malcolm W. Brown,et al.  Recognition memory: What are the roles of the perirhinal cortex and hippocampus? , 2001, Nature Reviews Neuroscience.

[66]  Aleksandra Shkurko Is social categorization based on relational ingroup/outgroup opposition? A meta-analysis. , 2013, Social cognitive and affective neuroscience.

[67]  S. Scott,et al.  Comprehension of familiar and unfamiliar native accents under adverse listening conditions. , 2009, Journal of experimental psychology. Human perception and performance.

[68]  Ingrid S. Johnsrude,et al.  Behavioral and fMRI evidence that cognitive ability modulates the effect of semantic context on speech intelligibility , 2012, Brain and Language.

[69]  Karl J. Friston,et al.  Analysis of fMRI Time-Series Revisited—Again , 1995, NeuroImage.

[70]  Steven L. Neuberg,et al.  A Continuum of Impression Formation, from Category-Based to Individuating Processes: Influences of Information and Motivation on Attention and Interpretation , 1990 .

[71]  D. Pisoni,et al.  Effects of talker, rate, and amplitude variation on recognition memory for spoken words , 1999, Perception & psychophysics.

[72]  Luciano Fadiga,et al.  When gaze opens the channel for communication: Integrative role of IFG and MPFC , 2015, NeuroImage.

[73]  Pascal Belin,et al.  Dorsal and Ventral Pathways for Prosody , 2015, Current Biology.

[74]  John D E Gabrieli,et al.  Assessing the influence of scanner background noise on auditory processing. I. An fMRI study comparing three experimental designs with varying degrees of scanner noise , 2007, Human brain mapping.

[75]  Matthew H. Davis,et al.  Neural dissociation in processing noise and accent in spoken language comprehension , 2012, Neuropsychologia.

[76]  M. Cikara,et al.  fMRI Repetition Suppression During Generalized Social Categorization , 2017, Scientific Reports.

[77]  Joan Y. Chiao,et al.  Intergroup Empathy: How Does Race Affect Empathic Neural Responses? , 2010, Current Biology.

[78]  M. Pell,et al.  Processing emotional tone from speech in Parkinson’s disease: A role for the basal ganglia , 2003, Cognitive, affective & behavioral neuroscience.

[79]  Isaac Siwale ON GLOBAL OPTIMIZATION , 2015 .

[80]  Daniel L. Schacter,et al.  Graded recall success: an event-related fMRI comparison of tip of the tongue and feeling of knowing , 2005, NeuroImage.

[81]  Shihui Han,et al.  Cultural experiences reduce racial bias in neural responses to others’ suffering , 2013 .

[82]  Jesper Andersson,et al.  Valid conjunction inference with the minimum statistic , 2005, NeuroImage.

[83]  Jörg Bahlmann,et al.  Predicting vocal emotion expressions from the human brain , 2013, Human brain mapping.

[84]  Didier Grandjean,et al.  Multiple subregions in superior temporal cortex are differentially sensitive to vocal expressions: A quantitative meta-analysis , 2013, Neuroscience & Biobehavioral Reviews.

[85]  N. Coupland,et al.  Ideologised values for British accents , 2007 .

[86]  M. Pell,et al.  The feeling of another's knowing: How "mixed messages" in speech are reconciled. , 2016, Journal of experimental psychology. Human perception and performance.

[87]  Marc Swerts,et al.  Neural coding of assessing another person's knowledge based on nonverbal cues. , 2015, Social cognitive and affective neuroscience.

[88]  Patrick C. M. Wong,et al.  Learning to recognize speakers of a non-native language: Implications for the functional organization of human auditory cortex , 2007, Neuropsychologia.

[89]  Rosina Lippi English with an Accent: Language, Ideology and Discrimination in the United States , 1997 .

[90]  M. Bresnahan,et al.  Attitudinal and affective response toward accented English , 2002 .

[91]  Yuta Katsumi,et al.  Neural Correlates of Racial Ingroup Bias in Observing Computer-Animated Social Encounters , 2018, Front. Hum. Neurosci..

[92]  H. Ackermann,et al.  Cerebral processing of linguistic and emotional prosody: fMRI studies. , 2006, Progress in brain research.

[93]  P. Belin,et al.  Thinking the voice: neural correlates of voice perception , 2004, Trends in Cognitive Sciences.

[94]  James M. McQueen,et al.  Neural mechanisms for voice recognition , 2010, NeuroImage.

[95]  Martin Meyer,et al.  Lateralization of emotional prosody in the brain: an overview and synopsis on the impact of study design. , 2006, Progress in brain research.

[96]  Matthew H. Davis,et al.  The neural mechanisms of speech comprehension: fMRI studies of semantic ambiguity. , 2005, Cerebral cortex.

[97]  R. Poldrack Inferring Mental States from Neuroimaging Data: From Reverse Inference to Large-Scale Decoding , 2011, Neuron.

[98]  Albert Costa,et al.  Using a Foreign Language Changes Our Choices , 2016, Trends in Cognitive Sciences.

[99]  Luis Sebastian Contreras-Huerta,et al.  Racial bias in neural response to others' pain is reduced with other-race contact , 2015, Cortex.

[100]  Joshua Correll,et al.  The impact of motivation on race-based impression formation , 2016, NeuroImage.

[101]  Lindsey A. Short,et al.  Speech Spectrum's Correlation with Speakers' Eysenck Personality Traits , 2012, PloS one.

[102]  R. Poldrack Can cognitive processes be inferred from neuroimaging data? , 2006, Trends in Cognitive Sciences.

[103]  Narun Pornpattananangkul,et al.  Cultural Neuroscience: Progress and Promise , 2013, Psychological inquiry.

[104]  Ting Zhang,et al.  Neural correlates of believing , 2017, NeuroImage.

[105]  Tracey M. Derwing,et al.  ACCENT, INTELLIGIBILITY, AND COMPREHENSIBILITY , 1997, Studies in Second Language Acquisition.

[106]  Tokiko Harada,et al.  Racial identification modulates default network activity for same and other races , 2012, Human brain mapping.

[107]  Henry S. Cheang,et al.  Social perception in adults with Parkinson's disease. , 2014, Neuropsychology.

[108]  Richard Futrell,et al.  Don’t Underestimate the Benefits of Being Misunderstood , 2017, Psychological science.

[109]  S. Fiske,et al.  Social groups that elicit disgust are differentially processed in mPFC. , 2007, Social cognitive and affective neuroscience.

[110]  Pascal Belin,et al.  A neural marker for social bias towards in-group accents , 2014 .

[111]  Negro dialect, ethnocentricism, and the distortion of information in the communicative process , 1972 .

[112]  H. Paterson,et al.  Low Vocal Pitch Preference Drives First Impressions Irrespective of Context in Male Voices but Not in Female Voices , 2016, Perception.

[113]  R. Zatorre,et al.  Human temporal-lobe response to vocal sounds. , 2002, Brain research. Cognitive brain research.

[114]  Fen Xu,et al.  Neural correlates of evaluations of lying and truth-telling in different social contexts , 2011, Brain Research.

[115]  Jonathan B Freeman,et al.  The neural origins of superficial and individuated judgments about ingroup and outgroup members , 2009, Human brain mapping.

[116]  Frank Van Overwalle,et al.  Understanding others' actions and goals by mirror and mentalizing systems: A meta-analysis , 2009, NeuroImage.

[117]  Timothy E. J. Behrens,et al.  Tools of the trade: psychophysiological interactions and functional connectivity. , 2012, Social cognitive and affective neuroscience.

[118]  A. Nobre,et al.  The Response of Left Temporal Cortex to Sentences , 2002, Journal of Cognitive Neuroscience.

[119]  R. Zatorre,et al.  Adaptation to speaker's voice in right anterior temporal lobe , 2003, Neuroreport.

[120]  J Mazziotta,et al.  A probabilistic atlas and reference system for the human brain: International Consortium for Brain Mapping (ICBM). , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[121]  Eric Hehman,et al.  The neural basis of stereotypic impact on multiple social categorization , 2014, NeuroImage.

[122]  Anne-Lise Giraud,et al.  Distinct functional substrates along the right superior temporal sulcus for the processing of voices , 2004, NeuroImage.

[123]  M. Pell,et al.  Neural responses towards a speaker's feeling of (un)knowing , 2016, Neuropsychologia.