How sensory-motor systems impact the neural organization for language: direct contrasts between spoken and signed language

To investigate the impact of sensory-motor systems on the neural organization for language, we conducted an H215O-PET study of sign and spoken word production (picture-naming) and an fMRI study of sign and audio-visual spoken language comprehension (detection of a semantically anomalous sentence) with hearing bilinguals who are native users of American Sign Language (ASL) and English. Directly contrasting speech and sign production revealed greater activation in bilateral parietal cortex for signing, while speaking resulted in greater activation in bilateral superior temporal cortex (STC) and right frontal cortex, likely reflecting auditory feedback control. Surprisingly, the language production contrast revealed a relative increase in activation in bilateral occipital cortex for speaking. We speculate that greater activation in visual cortex for speaking may actually reflect cortical attenuation when signing, which functions to distinguish self-produced from externally generated visual input. Directly contrasting speech and sign comprehension revealed greater activation in bilateral STC for speech and greater activation in bilateral occipital-temporal cortex for sign. Sign comprehension, like sign production, engaged bilateral parietal cortex to a greater extent than spoken language. We hypothesize that posterior parietal activation in part reflects processing related to spatial classifier constructions in ASL and that anterior parietal activation may reflect covert imitation that functions as a predictive model during sign comprehension. The conjunction analysis for comprehension revealed that both speech and sign bilaterally engaged the inferior frontal gyrus (with more extensive activation on the left) and the superior temporal sulcus, suggesting an invariant bilateral perisylvian language system. We conclude that surface level differences between sign and spoken languages should not be dismissed and are critical for understanding the neurobiology of language.

[1]  Cheryl M. Capek,et al.  The signing brain: the neurobiology of sign language , 2008, Trends in Cognitive Sciences.

[2]  Bencie Woll,et al.  Phonological processing in deaf signers and the impact of age of first language acquisition , 2008, NeuroImage.

[3]  A. Braun,et al.  Neural responses to meaningless pseudosigns: Evidence for sign-based phonetic processing in superior temporal cortex , 2011, Brain and Language.

[4]  Angela R. Laird,et al.  ALE meta-analysis of action observation and imitation in the human brain , 2010, NeuroImage.

[5]  Karen Emmorey,et al.  The Biology of Linguistic Expression Impacts Neural Correlates for Spatial Language , 2013, Journal of Cognitive Neuroscience.

[6]  Karen R. Dobkins,et al.  Visual Field Asymmetries for Motion Processing in Deaf and Hearing Signers , 2002, Brain and Cognition.

[7]  Karen Emmorey,et al.  Modulation of BOLD Response in Motion-sensitive Lateral Temporal Cortex by Real and Fictive Motion Sentences , 2010, Journal of Cognitive Neuroscience.

[8]  Karen Emmorey,et al.  The neural correlates of sign versus word production , 2007, NeuroImage.

[9]  Kevin P. Hinshaw,et al.  Functional Roles of Broca's Area and SMG: Evidence from Cortical Stimulation Mapping in a Deaf Signer , 1999, NeuroImage.

[10]  Riitta Salmelin,et al.  Subject's own speech reduces reactivity of the human auditory cortex , 1999, Neuroscience Letters.

[11]  K. Amunts,et al.  Broca's region , 2006 .

[12]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[13]  G. Hickok,et al.  Neural substrates for verbal working memory in deaf signers: fMRI study and lesion case report , 2005, Brain and Language.

[14]  Sarah H. Creem-Regehr,et al.  Sensory-motor and cognitive functions of the human posterior parietal cortex involved in manual actions , 2009, Neurobiology of Learning and Memory.

[15]  Jerker Rönnberg,et al.  Neural correlates of working memory for sign language. , 2004, Brain research. Cognitive brain research.

[16]  F. Guenther,et al.  A theoretical investigation of reference frames for the planning of speech movements. , 1998, Psychological review.

[17]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[18]  W. Sandler,et al.  Visual Intonation in the Prosody of a Sign Language , 2009, Language and speech.

[19]  M. Merzenich,et al.  Modulation of the Auditory Cortex during Speech: An MEG Study , 2002, Journal of Cognitive Neuroscience.

[20]  H. Damasio,et al.  A technique for neuroanatomical analysis of positron emission tomography images , 1993 .

[21]  Jason A. Tourville,et al.  Neural mechanisms underlying auditory feedback control of speech , 2008, NeuroImage.

[22]  Michael S. Beauchamp,et al.  FMRI group analysis combining effect estimates and their variances , 2012, NeuroImage.

[23]  D Bavelier,et al.  Cerebral organization for language in deaf and hearing subjects: biological constraints and effects of experience. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[24]  Richard D. Hichwa,et al.  Clinical Blood Flow Measurement with [15O] Water and Positron Emission Tomography (PET) , 1995 .

[25]  Craig J. Brozinsky,et al.  Impact of Early Deafness and Early Exposure to Sign Language on the Cerebral Organization for Motion Processing , 2001, The Journal of Neuroscience.

[26]  Cathy J. Price,et al.  A review and synthesis of the first 20 years of PET and fMRI studies of heard speech, spoken language and reading , 2012, NeuroImage.

[27]  R. J. Frank,et al.  Reliability of PET activation across statistical methods, subject groups, and sample sizes , 1996, Human brain mapping.

[28]  Kayoko Okada,et al.  Area Spt in the Human Planum Temporale Supports Sensory-motor Integration for Speech Processing Establishing the Existence of Distinct Sen- Sory versus Motor Activation Patterns Would Establish That , 2022 .

[29]  M. Schlesewsky,et al.  Lexical prediction via forward models: N400 evidence from German Sign Language , 2013, Neuropsychologia.

[30]  Alan J. Miller,et al.  Least Squares Routines to Supplement Those of Gentleman , 1992 .

[31]  D Bavelier,et al.  Encoding, rehearsal, and recall in signers and speakers: shared network but differential engagement. , 2008, Cerebral cortex.

[32]  A. Braun,et al.  Activation of Broca’s area during the production of spoken and signed language: a combined cytoarchitectonic mapping and PET analysis , 2003, Neuropsychologia.

[33]  Frank H Guenther,et al.  The DIVA model: A neural theory of speech acquisition and production , 2011, Language and cognitive processes.

[34]  A. Braun,et al.  The neural organization of discourse: an H2 15O-PET study of narrative production in English and American sign language. , 2001, Brain : a journal of neurology.

[35]  M. Seghier,et al.  Where, When and Why Brain Activation Differs for Bilinguals and Monolinguals during Picture Naming and Reading Aloud , 2011, Cerebral cortex.

[36]  D. Corina,et al.  Cross-Linguistic Differences in the Neural Representation of Human Language: Evidence from Users of Signed Languages , 2013, Front. Psychology.

[37]  M. Raichle,et al.  Brain blood flow measured with intravenous H2(15)O. I. Theory and error analysis. , 1983, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[38]  D. Corina,et al.  Sign Language Processing and the Mirror Neuron System , 2006, Cortex.

[39]  J C Mazziotta,et al.  Creation and use of a Talairach‐compatible atlas for accurate, automated, nonlinear intersubject registration, and analysis of functional imaging data , 1999, Human brain mapping.

[40]  Christian Keysers,et al.  Mapping the flow of information within the putative mirror neuron system during gesture observation , 2011, NeuroImage.

[41]  Karen Emmorey,et al.  The influence of visual feedback and register changes on sign language production: A kinematic study with deaf signers. , 2009, Applied psycholinguistics.

[42]  V. Jousmäki,et al.  Attenuation of somatosensory responses to self-produced tactile stimulation. , 2010, Cerebral cortex.

[43]  K. Emmorey.  The Neurobiology of Sign Language , 2015 .

[44]  Alan C. Evans,et al.  Speech-like cerebral activity in profoundly deaf people processing signed languages: implications for the neural basis of human language. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Leslie G. Ungerleider,et al.  Effect of task difficulty on cerebral blood flow during perceptual matching of faces , 1996, Human brain mapping.

[46]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[47]  K. Emmorey,et al.  Visual feedback and self-monitoring of sign language. , 2009, Journal of memory and language.

[48]  R. J. Frank,et al.  Brainvox: An Interactive, Multimodal Visualization and Analysis System for Neuroanatomical Imaging , 1997, NeuroImage.

[49]  A. Postma,et al.  Production and Detection of Speech Errors in Silent, Mouthed, Noise-Masked, and Normal Auditory Feedback Speech , 1996 .

[50]  Kara D. Federmeier,et al.  Timed picture naming in seven languages , 2003, Psychonomic bulletin & review.

[51]  P. Indefrey,et al.  The cognitive neuroscience of second language acquisition , 2006 .

[52]  James M. Kilner,et al.  More than one pathway to action understanding , 2011, Trends in Cognitive Sciences.

[53]  J Rönnberg,et al.  Signed and spoken language perception studied by positron emission tomography , 1997, Neurology.

[54]  Kuniyoshi L Sakai,et al.  Sign and speech: amodal commonality in left hemisphere dominance for comprehension of sentences. , 2005, Brain : a journal of neurology.

[55]  K. Emmorey,et al.  Differential Processing of Topographic and Referential Functions of Space , 2013 .

[56]  Ursula Bellugi,et al.  Neural Organization of Linguistic Short-term Memory is Sensory Modalitydependent: Evidence from Signed and Spoken Language , 2008, Journal of Cognitive Neuroscience.

[57]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[58]  Peter Hagoort,et al.  MUC (Memory, Unification, Control) and beyond , 2013, Front. Psychol..

[59]  A. Friederici.  The brain basis of language processing: from structure to function. , 2011, Physiological reviews.

[60]  Ursula Bellugi,et al.  What's right about the neural organization of sign language? A perspective on recent neuroimaging results , 1998, Trends in Cognitive Sciences.

[61]  Alan C. Evans,et al.  A Three-Dimensional Statistical Analysis for CBF Activation Studies in Human Brain , 1992, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[62]  Bencie Woll,et al.  Testing comprehension abilities in users of British Sign Language following CVA , 2005, Brain and Language.

[63]  Jesper Andersson,et al.  Valid conjunction inference with the minimum statistic , 2005, NeuroImage.

[64]  Stephen M. Smith,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[65]  G. Rizzolatti,et al.  Action observation activates premotor and parietal areas in a somatotopic manner: an fMRI study , 2001, The European journal of neuroscience.

[66]  Ursula Bellugi,et al.  The neurobiology of sign language and its implications for the neural basis of language , 1996, Nature.

[67]  Karen Emmorey,et al.  journal homepage: www.elsevier.com/locate/b&l The bimodal bilingual brain: Effects of sign language experience q , 2022 .

[68]  W. Sandler.  Prosody in Two Natural Language Modalities * , 1999 .

[69]  K. Worsley,et al.  Local Maxima and the Expected Euler Characteristic of Excursion Sets of χ 2, F and t Fields , 1994, Advances in Applied Probability.

[70]  D. Brentari,et al.  Amodal Aspects of Linguistic Design , 2013, PloS one.

[71]  Karen Emmorey,et al.  The neural correlates of spatial language in English and American Sign Language: a PET study with hearing bilinguals , 2005, NeuroImage.

[72]  R. Wise,et al.  Two Tongues, One Brain: Imaging Bilingual Speech Production , 2011, Front. Psychology.

[73]  Karen Emmorey,et al.  The use of visual feedback during signing: evidence from signers with impaired vision. , 2009, Journal of deaf studies and deaf education.

[74]  Bertram Walter,et al.  Your mind's hand: Motor imagery of pointing movements with different accuracy , 2010, NeuroImage.

[75]  P. McGuire,et al.  Neural systems underlying British Sign Language and audio-visual English processing in native users. , 2002, Brain : a journal of neurology.

[76]  Allen R. Braun,et al.  Language Lateralization in a Bimanual Language , 2003, Journal of Cognitive Neuroscience.

[77]  H. Damasio,et al.  Validation of Partial Tissue Segmentation of Single-Channel Magnetic Resonance Images of the Brain , 2000, NeuroImage.

[78]  W. Levelt,et al.  The spatial and temporal signatures of word production components , 2004, Cognition.

[79]  Peter Erhard,et al.  Involvement of classical anterior and posterior language areas in sign language production, as investigated by 4 T functional magnetic resonance imaging , 2004, Neuroscience Letters.

[80]  Karen Emmorey,et al.  Motion-sensitive cortex and motion semantics in American Sign Language , 2012, NeuroImage.

[81]  C. Fiebach,et al.  The role of left inferior frontal and superior temporal cortex in sentence comprehension: localizing syntactic and semantic processes. , 2003, Cerebral cortex.

[82]  Eric Halgren,et al.  Signed Words in the Congenitally Deaf Evoke Typical Late Lexicosemantic Responses with No Early Visual Responses in Left Superior Temporal Cortex , 2012, The Journal of Neuroscience.

[83]  Bencie Woll,et al.  Lexical and sentential processing in British Sign Language , 2006, Human brain mapping.

[84]  Aaron J. Newman,et al.  Prosodic and narrative processing in American Sign Language: An fMRI study , 2010, NeuroImage.

[85]  Philip K. McGuire,et al.  Neural Correlates of British Sign Language Comprehension: Spatial Processing Demands of Topographic Language , 2002, Journal of Cognitive Neuroscience.

[86]  David F. Armstrong.  Language, Cognition, and the Brain: Insights from Sign Langauge Research (review) , 2003 .

[87]  Karen Emmorey,et al.  Neural systems underlying lexical retrieval for sign language , 2003, Neuropsychologia.

[88]  Kuncheng Li,et al.  Brain activations associated with sign production using word and picture inputs in deaf signers , 2011, Brain and Language.

[89]  M. Pickering,et al.  An integrated theory of language production and comprehension. , 2013, The Behavioral and brain sciences.

[90]  P. Hervé,et al.  A common neural system is activated in hearing non-signers to process French Sign language and spoken French , 2011, Brain Research Bulletin.

[91]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[92]  Ione Fine,et al.  Visual stimuli activate auditory cortex in the deaf , 2001, Nature Neuroscience.

[93]  H. Damasio,et al.  A new technique for pet slice orientation and MRI‐PET coregistration , 1994 .

[94]  Cheryl M. Capek,et al.  Dissociating cognitive and sensory neural plasticity in human superior temporal cortex , 2013, Nature Communications.