Convergence of spoken and written language processing in the superior temporal sulcus

&NA; Spoken and written language processing streams converge in the superior temporal sulcus (STS), but the functional and anatomical nature of this convergence is not clear. We used functional MRI to quantify neural responses to spoken and written language, along with unintelligible stimuli in each modality, and employed several strategies to segregate activations on the dorsal and ventral banks of the STS. We found that intelligible and unintelligible inputs in both modalities activated the dorsal bank of the STS. The posterior dorsal bank was able to discriminate between modalities based on distributed patterns of activity, pointing to a role in encoding of phonological and orthographic word forms. The anterior dorsal bank was agnostic to input modality, suggesting that this region represents abstract lexical nodes. In the ventral bank of the STS, responses to unintelligible inputs in both modalities were attenuated, while intelligible inputs continued to drive activation, indicative of higher level semantic and syntactic processing. Our results suggest that the processing of spoken and written language converges on the posterior dorsal bank of the STS, which is the first of a heterogeneous set of language regions within the STS, with distinct functions spanning a broad range of linguistic processes. HighlightsA functional parcellation of the superior temporal sulcus is proposed.The posterior dorsal bank encodes phonological and orthographic word forms.This is the site where spoken and written language processing streams first converge.The anterior dorsal bank represents abstract lexical nodes.Several ventral bank regions are involved in higher level linguistic processing.

[1]  Gary H Glover,et al.  Calibration of BOLD fMRI using breath holding reduces group variance during a cognitive task , 2007, Human brain mapping.

[2]  T. Powell,et al.  An anatomical study of converging sensory pathways within the cerebral cortex of the monkey. , 1970, Brain : a journal of neurology.

[3]  R. Zatorre,et al.  Adaptation to speaker's voice in right anterior temporal lobe , 2003, Neuroreport.

[4]  S. Dehaene,et al.  The unique role of the visual word form area in reading , 2011, Trends in Cognitive Sciences.

[5]  A. Ishai,et al.  Distributed and Overlapping Representations of Faces and Objects in Ventral Temporal Cortex , 2001, Science.

[6]  Charles E. Schroeder,et al.  Feedforward and feedback projections of caudal belt and parabelt areas of auditory cortex: refining the hierarchical model , 2014, Front. Neurosci..

[7]  Gregory Hickok,et al.  Phonological repetition-suppression in bilateral superior temporal sulci , 2010, NeuroImage.

[8]  Jonathan H. Venezia,et al.  Hierarchical organization of human auditory cortex: evidence from acoustic invariance in the response to intelligible speech. , 2010, Cerebral cortex.

[9]  Maria Luisa Gorno-Tempini,et al.  Connected speech production in three variants of primary progressive aphasia. , 2010, Brain : a journal of neurology.

[10]  Elia Formisano,et al.  An anatomical and functional topography of human auditory cortical areas , 2014, Front. Neurosci..

[11]  B. Miller,et al.  Variable disruption of a syntactic processing network in primary progressive aphasia. , 2016, Brain : a journal of neurology.

[12]  Grant M. Walker,et al.  Anterior temporal involvement in semantic word retrieval: voxel-based lesion-symptom mapping evidence from aphasia. , 2009, Brain : a journal of neurology.

[13]  S. Scott,et al.  Identification of a pathway for intelligible speech in the left temporal lobe. , 2000, Brain : a journal of neurology.

[14]  Alan C. Evans,et al.  A General Statistical Analysis for fMRI Data , 2000, NeuroImage.

[15]  H. Coslett,et al.  Localization of sublexical speech perception components , 2010, Brain and Language.

[16]  Miranda Robertson,et al.  Neural systems , 1977, Nature.

[17]  Leslie G. Ungerleider,et al.  Multiple visual areas in the caudal superior temporal sulcus of the macaque , 1986, The Journal of comparative neurology.

[18]  Keith Johnson,et al.  Phonetic Feature Encoding in Human Superior Temporal Gyrus , 2014, Science.

[19]  Michael Petrides,et al.  The morphology and variability of the caudal rami of the superior temporal sulcus , 2012, The European journal of neuroscience.

[20]  A. Friederici,et al.  The role of the posterior superior temporal cortex in sentence comprehension , 2009, Neuroreport.

[21]  S. Aronson,et al.  AN ANATOMICAL STUDY , 1961 .

[22]  T. Rogers,et al.  Where do you know what you know? The representation of semantic knowledge in the human brain , 2007, Nature Reviews Neuroscience.

[23]  J. Binder,et al.  The topography of callosal reading pathways. A case-control analysis. , 1992, Brain : a journal of neurology.

[24]  R. Goebel,et al.  Mirror-Symmetric Tonotopic Maps in Human Primary Auditory Cortex , 2003, Neuron.

[25]  Kevin Murphy,et al.  Robustly measuring vascular reactivity differences with breath-hold: Normalising stimulus-evoked and resting state BOLD fMRI data , 2011, NeuroImage.

[26]  Stanislas Dehaene,et al.  A Temporal Bottleneck in the Language Comprehension Network , 2012, The Journal of Neuroscience.

[27]  James L. McClelland,et al.  The TRACE model of speech perception , 1986, Cognitive Psychology.

[28]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[29]  Mariano Sigman,et al.  Hierarchical Coding of Letter Strings in the Ventral Stream: Dissecting the Inner Organization of the Visual Word-Form System , 2007, Neuron.

[30]  Kayoko Okada,et al.  Identification of lexical–phonological networks in the superior temporal sulcus using functional magnetic resonance imaging , 2006, Neuroreport.

[31]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[32]  Matthew H. Davis,et al.  Hierarchical Processing in Spoken Language Comprehension , 2003, The Journal of Neuroscience.

[33]  Steven Z. Rapcsak,et al.  The role of left posterior inferior temporal cortex in spelling , 2004, Neurology.

[34]  C. Price,et al.  The Interactive Account of ventral occipitotemporal contributions to reading , 2011, Trends in Cognitive Sciences.

[35]  William W. Graves,et al.  Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. , 2009, Cerebral cortex.

[36]  S. Dehaene,et al.  Cortical representation of the constituent structure of sentences , 2011, Proceedings of the National Academy of Sciences.

[37]  Thomas L. Griffiths,et al.  Supplementary Information for Natural Speech Reveals the Semantic Maps That Tile Human Cerebral Cortex , 2022 .

[38]  B. Miller,et al.  Neural Correlates of Syntactic Processing in the Nonfluent Variant of Primary Progressive Aphasia , 2010, The Journal of Neuroscience.

[39]  Robert Lindenberg,et al.  Supramodal language comprehension: Role of the left temporal lobe for listening and reading , 2007, Neuropsychologia.

[40]  Dawn G. Blasko,et al.  Do the Beginnings of Spoken Words Have a Special Status in Auditory Word Recognition , 1993 .

[41]  S. Scott,et al.  The Pathways for Intelligible Speech: Multivariate and Univariate Perspectives , 2013, Cerebral cortex.

[42]  B. Seltzer,et al.  Chemoarchitectonics and corticocortical terminations within the superior temporal sulcus of the rhesus monkey: Evidence for subdivisions of superior temporal polysensory cortex , 1995, The Journal of comparative neurology.

[43]  James R. Booth,et al.  Functional Anatomy of Intra- and Cross-Modal Lexical Tasks , 2002, NeuroImage.

[44]  Grant M. Walker,et al.  Neuroanatomical dissociation for taxonomic and thematic knowledge in the human brain , 2011, Proceedings of the National Academy of Sciences.

[45]  Jiang Xu,et al.  Language in context: emergent features of word, sentence, and narrative comprehension , 2005, NeuroImage.

[46]  D. Tank,et al.  4 Tesla gradient recalled echo characteristics of photic stimulation‐induced signal changes in the human primary visual cortex , 1993 .

[47]  M. D’Esposito,et al.  Reducing vascular variability of fMRI data across aging populations using a breathholding task , 2007, Human brain mapping.

[48]  Colin Humphries,et al.  The functional organization of the left STS: a large scale meta-analysis of PET and fMRI studies of healthy adults , 2014, Front. Neurosci..

[49]  Michael S. Beauchamp,et al.  A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion , 2012, NeuroImage.

[50]  J. Rauschecker,et al.  Vowel sound extraction in anterior superior temporal cortex , 2006, Human brain mapping.

[51]  David A. Medler,et al.  Cerebral Cortex doi:10.1093/cercor/bhi040 Cerebral Cortex Advance Access published February 9, 2005 , 2022 .

[52]  Christopher J. Honey,et al.  Selective and Invariant Neural Responses to Spoken and Written Narratives , 2013, The Journal of Neuroscience.

[53]  Daniel S. Margulies,et al.  Prioritizing spatial accuracy in high-resolution fMRI data using multivariate feature weight mapping , 2014, Front. Neurosci..

[54]  A. Kertesz,et al.  Localization in transcortical sensory aphasia. , 1982, Archives of neurology.

[55]  C. M. Sutherland Computed tomographic scan. , 1982, Archives of internal medicine.

[56]  E. Chang,et al.  Categorical Speech Representation in Human Superior Temporal Gyrus , 2010, Nature Neuroscience.

[57]  C. J. Price,et al.  Cortical localisation of the visual and auditory word form areas: A reconsideration of the evidence , 2003, Brain and Language.

[58]  Cyril Poupon,et al.  Anatomical Connections of the Visual Word Form Area , 2014, The Journal of Neuroscience.

[59]  J. A. Frost,et al.  Function of the left planum temporale in auditory and linguistic processing , 1996, NeuroImage.

[60]  J. A. Frost,et al.  Conceptual Processing during the Conscious Resting State: A Functional MRI Study , 1999, Journal of Cognitive Neuroscience.

[61]  Thomas E. Nichols,et al.  Nonparametric permutation tests for functional neuroimaging: A primer with examples , 2002, Human brain mapping.

[62]  Thomas E. Nichols,et al.  Power calculation for group fMRI studies accounting for arbitrary design and temporal autocorrelation , 2008, NeuroImage.

[63]  Francisco Pereira,et al.  Information mapping with pattern classifiers: A comparative study , 2011, NeuroImage.

[64]  D. Knopman,et al.  Computed tomographic scan correlates of auditory comprehension deficits in aphasia: A prospective recovery study , 1983, Annals of neurology.

[65]  Mallorie Leinenger Phonological coding during reading. , 2014, Psychological bulletin.

[66]  E. T. Possing,et al.  Human temporal lobe activation by speech and nonspeech sounds. , 2000, Cerebral cortex.

[67]  James L. McClelland,et al.  No Right to Speak? The Relationship between Object Naming and Semantic Impairment:Neuropsychological Evidence and a Computational Model , 2001, Journal of Cognitive Neuroscience.

[68]  S. Heim,et al.  Identifying brain systems for gaze orienting during reading: fMRI investigation of the Landolt paradigm , 2013, Front. Hum. Neurosci..

[69]  Robert Leech,et al.  The contribution of the inferior parietal cortex to spoken language production , 2012, Brain and Language.

[70]  Elizabeth Jefferies,et al.  Both the Middle Temporal Gyrus and the Ventral Anterior Temporal Area Are Crucial for Multimodal Semantic Processing: Distortion-corrected fMRI Evidence for a Double Gradient of Information Convergence in the Temporal Lobes , 2012, Journal of Cognitive Neuroscience.

[71]  W. Levelt,et al.  The spatial and temporal signatures of word production components , 2004, Cognition.

[72]  J. Binder,et al.  The new neuroanatomy of speech perception. , 2000, Brain : a journal of neurology.

[73]  Matthew H. Davis,et al.  Can cognitive models explain brain activation during word and pseudoword reading? A meta-analysis of 36 neuroimaging studies. , 2013, Psychological bulletin.

[74]  N. Geschwind Disconnexion syndromes in animals and man. I. , 1965, Brain : a journal of neurology.

[75]  J. Hodges,et al.  Semantic dementia. Progressive fluent aphasia with temporal lobe atrophy. , 1992, Brain : a journal of neurology.

[76]  Stephen M Smith,et al.  Correspondence of the brain's functional architecture during activation and rest , 2009, Proceedings of the National Academy of Sciences.

[77]  C. Price,et al.  The role of the posterior superior temporal sulcus in audiovisual processing. , 2008, Cerebral cortex.

[78]  B. Seltzer,et al.  Architectonics and cortical connections of the upper bank of the superior temporal sulcus in the rhesus monkey: An analysis in the tangential plane , 2003, The Journal of comparative neurology.

[79]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[80]  S. Lupker,et al.  Does jugde activate COURT? Transposed-letter similarity effects in masked associative priming , 2003, Memory & cognition.

[81]  Karl J. Friston,et al.  Statistical parametric mapping , 2013 .

[82]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[83]  A. Turken,et al.  The Neural Architecture of the Language Comprehension Network: Converging Evidence from Lesion and Connectivity Analyses , 2011, Front. Syst. Neurosci..

[84]  D. Pandya,et al.  Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey , 1978, Brain Research.

[85]  S. Scott,et al.  Converging Language Streams in the Human Temporal Lobe , 2006, The Journal of Neuroscience.

[86]  G. Rees Statistical Parametric Mapping , 2004, Practical Neurology.

[87]  Mikko Sams,et al.  Perceiving identical sounds as speech or non-speech modulates activity in the left posterior superior temporal sulcus , 2006, NeuroImage.

[88]  Stephen M. Wilson,et al.  Lexical access in semantic variant PPA: Evidence for a post-semantic contribution to naming deficits , 2017, Neuropsychologia.

[89]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[90]  J. Rauschecker,et al.  Maps and streams in the auditory cortex: nonhuman primates illuminate human speech processing , 2009, Nature Neuroscience.

[91]  Stephen M. Wilson,et al.  The impact of vascular factors on language localization in the superior temporal sulcus , 2014, Human brain mapping.

[92]  Karl J. Friston,et al.  The cortical localization of the lexicons. Positron emission tomography evidence. , 1992, Brain : a journal of neurology.

[93]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[94]  E C Wong,et al.  A hypercapnia‐based normalization method for improved spatial localization of human brain activation with fMRI , 1997, NMR in biomedicine.

[95]  K. Strimmer,et al.  Statistical Applications in Genetics and Molecular Biology A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics , 2011 .

[96]  Malathi Thothathiri,et al.  The Neural Basis of Reversible Sentence Comprehension: Evidence from Voxel-based Lesion Symptom Mapping in Aphasia , 2012, Journal of Cognitive Neuroscience.

[97]  R. Goebel,et al.  Integration of Letters and Speech Sounds in the Human Brain , 2004, Neuron.

[98]  E. Haacke,et al.  Identification of vascular structures as a major source of signal contrast in high resolution 2D and 3D functional activation imaging of the motor cortex at l.5T preliminary results , 1993, Magnetic resonance in medicine.

[99]  William W. Graves,et al.  Neural Systems for Reading Aloud: A Multiparametric Approach , 2009, Cerebral cortex.

[100]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[101]  Bernard Mazoyer,et al.  Impact of modality and linguistic complexity during reading and listening tasks , 2007, NeuroImage.

[102]  M. Greenlee,et al.  Combined diffusion-weighted and functional magnetic resonance imaging reveals a temporal-occipital network involved in auditory-visual object processing , 2013, Front. Integr. Neurosci..

[103]  Chandan J. Vaidya,et al.  Functional anatomy of listening and reading comprehension during development , 2010, Brain and Language.

[104]  J. Rauschecker,et al.  Phoneme and word recognition in the auditory ventral stream , 2012, Proceedings of the National Academy of Sciences.

[105]  Arno Klein,et al.  Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration , 2009, NeuroImage.

[106]  Roy D. Patterson,et al.  Locating the initial stages of speech–sound processing in human temporal cortex , 2006, NeuroImage.

[107]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[108]  Matthew H. Davis,et al.  Interpreting response time effects in functional imaging studies , 2014, NeuroImage.

[109]  G I de Zubicaray,et al.  The semantic interference effect in the picture‐word paradigm: An event‐related fMRI study employing overt responses , 2001, Human brain mapping.

[110]  Rutvik H. Desai,et al.  The neural substrates of natural reading: a comparison of normal and nonword text using eyetracking and fMRI , 2014, Front. Hum. Neurosci..

[111]  Anjan Chatterjee,et al.  The Functional Neuroanatomy of Thematic Role and Locative Relational Knowledge , 2007, Journal of Cognitive Neuroscience.

[112]  S Lehéricy,et al.  The visual word form area: spatial and temporal characterization of an initial stage of reading in normal subjects and posterior split-brain patients. , 2000, Brain : a journal of neurology.

[113]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[114]  Egill Rostrup,et al.  Hypercapnic normalization of BOLD fMRI: comparison across field strengths and pulse sequences , 2004, NeuroImage.

[115]  Christian Beckmann,et al.  The Visual Word Form System in Context , 2011, The Journal of Neuroscience.

[116]  John Ashburner,et al.  A fast diffeomorphic image registration algorithm , 2007, NeuroImage.