UvA-DARE (Digital Academic Repository) Unification of speaker and meaning in language comprehension: an fMRI study

When interpreting a message, a listener takes into account several sources of linguistic and extralinguistic information. Here we focused on one particular form of extralinguistic information, certain speaker characteristics as conveyed by the voice. Using functional magnetic resonance imaging, we examined the neural structures involved in the unification of sentence meaning and voice-based inferences about the speaker's age, sex, or social background. We found enhanced activation in the inferior frontal gyrus bilaterally (BA 45/47) during listening to sentences whose meaning was incongruent with inferred speaker characteristics. Furthermore, our results showed an overlap in brain regions involved in unification of speaker-related information and those used for the unification of semantic and world knowledge information [inferior frontal gyrus bilaterally (BA 45/47) and left middle temporal gyrus (BA 21)]. These findings provide evidence for a shared neural unification system for linguistic and extralinguistic sources of information and extend the existing knowledge about the role of inferior frontal cortex as a crucial component for unification during language comprehension.

[1]  Roel M. Willems,et al.  Add a picture for suspense: neural correlates of the interaction between language and visual information in the perception of fear. , 2011, Social cognitive and affective neuroscience.

[2]  D. Poeppel,et al.  Towards a functional neuroanatomy of speech perception , 2000, Trends in Cognitive Sciences.

[3]  A. Dale,et al.  Distinct Patterns of Neural Modulation during the Processing of Conceptual and Syntactic Anomalies , 2003, Journal of Cognitive Neuroscience.

[4]  Remco J. Renken,et al.  Semantic ambiguity processing in sentence context: Evidence from event-related fMRI , 2007, NeuroImage.

[5]  R. Poldrack,et al.  Dissociable Controlled Retrieval and Generalized Selection Mechanisms in Ventrolateral Prefrontal Cortex , 2005, Neuron.

[6]  S. Scott,et al.  Identification of a pathway for intelligible speech in the left temporal lobe. , 2000, Brain : a journal of neurology.

[7]  Peter Hagoort,et al.  The Neural Integration of Speaker and Message , 2008, Journal of Cognitive Neuroscience.

[8]  D. Shankweiler,et al.  An Event-related Neuroimaging Study Distinguishing Form and Content in Sentence Processing , 2000, Journal of Cognitive Neuroscience.

[9]  Christine Chiarello,et al.  Sentence context and lexical ambiguity resolution by the two hemispheres , 1998, Neuropsychologia.

[10]  R. Zatorre,et al.  Adaptation to speaker's voice in right anterior temporal lobe , 2003, Neuroreport.

[11]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[12]  M. Greicius,et al.  Default-Mode Activity during a Passive Sensory Task: Uncoupled from Deactivation but Impacting Activation , 2004, Journal of Cognitive Neuroscience.

[13]  Anne-Lise Giraud,et al.  Distinct functional substrates along the right superior temporal sulcus for the processing of voices , 2004, NeuroImage.

[14]  E. Bizzi,et al.  The Cognitive Neurosciences , 1996 .

[15]  Rainer Goebel,et al.  The effect of temporal asynchrony on the multisensory integration of letters and speech sounds. , 2006, Cerebral cortex.

[16]  R. Zatorre,et al.  Voice-selective areas in human auditory cortex , 2000, Nature.

[17]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[18]  Simon B. Eickhoff,et al.  A new SPM toolbox for combining probabilistic cytoarchitectonic maps and functional imaging data , 2005, NeuroImage.

[19]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[20]  P. Hagoort.  On Broca, brain, and binding: a new framework , 2005, Trends in Cognitive Sciences.

[21]  P Hagoort,et al.  Semantic, factual, and social language comprehension in adolescents with autism: an FMRI study. , 2010, Cerebral cortex.

[22]  Matthew H. Davis,et al.  Hierarchical Processing in Spoken Language Comprehension , 2003, The Journal of Neuroscience.

[23]  M. Jung-Beeman.  Bilateral brain processes for comprehending natural language , 2005, Trends in Cognitive Sciences.

[24]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[25]  Denis Dooley,et al.  Atlas of the Human Brain. , 1971 .

[26]  G. Kempen,et al.  Syntactic structure assembly in human parsing: a computational model based on competitive inhibition and a lexicalist grammar , 2000, Cognition.

[27]  Anne Cutler,et al.  Prelexical and lexical processing in listening , 2004 .

[28]  Evelyn C. Ferstl,et al.  The extended language network: A meta‐analysis of neuroimaging studies on text comprehension , 2008, Human brain mapping.

[29]  M. Just,et al.  Brain Activation Modulated by Sentence Comprehension , 1996, Science.

[30]  P. Belin,et al.  Thinking the voice: neural correlates of voice perception , 2004, Trends in Cognitive Sciences.

[31]  Karl Magnus Petersson,et al.  Artificial syntactic violations activate Broca's region , 2004, Cogn. Sci..

[32]  Peter Hagoort,et al.  Brain responses to lexical ambiguity resolution and parsing. , 1994 .

[33]  Marcel Adam Just,et al.  From the SelectedWorks of Marcel Adam Just 2007 Lexical ambiguity in sentence comprehension , 2016 .

[34]  Roel M. Willems,et al.  When language meets action: the neural integration of gesture and speech. , 2007, Cerebral cortex.

[35]  S. Bookheimer,et al.  Form and Content Dissociating Syntax and Semantics in Sentence Comprehension , 1999, Neuron.

[36]  Weijia Ni,et al.  Sentence complexity and input modality effects in sentence comprehension: an fMRI study , 2004, NeuroImage.

[37]  S. Bookheimer.  Functional MRI of language: new approaches to understanding the cortical organization of semantic processing. , 2002, Annual review of neuroscience.

[38]  Peter Hagoort,et al.  When Elephants Fly: Differential Sensitivity of Right and Left Inferior Frontal Gyri to Discourse and World Knowledge , 2009, Journal of Cognitive Neuroscience.

[39]  Ray Jackendoff,et al.  A Parallel Architecture perspective on language processing , 2007, Brain Research.

[40]  Colin Humphries,et al.  Syntactic and Semantic Modulation of Neural Activity during Auditory Sentence Comprehension , 2006, Journal of Cognitive Neuroscience.

[41]  Gina R. Kuperberg,et al.  Neuroanatomical distinctions within the semantic system during sentence comprehension: Evidence from functional magnetic resonance imaging , 2008, NeuroImage.

[42]  Pascal Belin,et al.  Is voice processing species-specific in human auditory cortex? An fMRI study , 2004, NeuroImage.

[43]  R. Zatorre,et al.  Human temporal-lobe response to vocal sounds. , 2002, Brain research. Cognitive brain research.

[44]  M. Farah,et al.  Role of left inferior prefrontal cortex in retrieval of semantic knowledge: a reevaluation. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Matthew H. Davis,et al.  The neural mechanisms of speech comprehension: fMRI studies of semantic ambiguity. , 2005, Cerebral cortex.

[46]  Roel M. Willems,et al.  Differential roles for left inferior frontal and superior temporal cortex in multimodal integration of action and language , 2009, NeuroImage.

[47]  Peter Hagoort,et al.  Effective connectivity of cortical and subcortical regions during unification of sentence structure , 2010, NeuroImage.

[48]  Istvan Molnar-Szakacs,et al.  Beyond superior temporal cortex: intersubject correlations in narrative speech comprehension. , 2008, Cerebral cortex.

[49]  D. A. Robertson,et al.  Functional Neuroanatomy of the Cognitive Process of Mapping During Discourse Comprehension , 2000, Psychological science.

[50]  M. Torrens.  Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[51]  J L Lancaster,et al.  Automated Talairach Atlas labels for functional brain mapping , 2000, Human brain mapping.

[52]  M. Gazzaniga,et al.  The new cognitive neurosciences , 2000 .

[53]  Kathryn Campbell-Kibler,et al.  Sociolinguistics and Perception , 2010, Lang. Linguistics Compass.

[54]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[55]  W. Kintsch,et al.  Strategies of discourse comprehension , 1983 .

[56]  M. Raichle,et al.  Searching for a baseline: Functional imaging and the resting human brain , 2001, Nature Reviews Neuroscience.

[57]  A. Kleinschmidt,et al.  Modulation of neural responses to speech by directing attention to voices or verbal content. , 2003, Brain research. Cognitive brain research.

[58]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[59]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[60]  G L Shulman,et al.  INAUGURAL ARTICLE by a Recently Elected Academy Member:A default mode of brain function , 2001 .

[61]  K. Amunts,et al.  Brodmann's Areas 17 and 18 Brought into Stereotaxic Space—Where and How Variable? , 2000, NeuroImage.

[62]  Evelyn C. Ferstl,et al.  Emotional and Temporal Aspects of Situation Model Processing during Text Comprehension: An Event-Related fMRI Study , 2005, Journal of Cognitive Neuroscience.

[63]  P. Hagoort,et al.  Integration of Word Meaning and World Knowledge in Language Comprehension , 2004, Science.

[64]  C. Curtis,et al.  Persistent activity in the prefrontal cortex during working memory , 2003, Trends in Cognitive Sciences.

[65]  Phillip J. Holcomb,et al.  Making sense of discourse: An fMRI study of causal inferencing across sentences , 2006, NeuroImage.

[66]  D. Schacter,et al.  The Brain's Default Network , 2008, Annals of the New York Academy of Sciences.

[67]  M. Kutas,et al.  Semantic integration in reading: engagement of the right hemisphere during discourse processing. , 1999, Brain : a journal of neurology.

[68]  E. G. Jones.  Cerebral Cortex , 1987, Cerebral Cortex.

[69]  Mante S. Nieuwland,et al.  On sense and reference: Examining the functional neuroanatomy of referential processing , 2007, NeuroImage.