Shared neural correlates for building phrases in signed and spoken language

Research on the mental representation of human language has convincingly shown that sign languages are structured similarly to spoken languages. However, whether the same neurobiology underlies the online construction of complex linguistic structures in sign and speech remains unknown. To investigate this question with maximally controlled stimuli, we studied the production of minimal two-word phrases in sign and speech. Signers and speakers viewed the same pictures during magnetoencephalography recording and named them with semantically identical expressions. For both signers and speakers, phrase building engaged left anterior temporal and ventromedial cortices with similar timing, despite different linguistic articulators. Thus the neurobiological similarity of sign and speech goes beyond gross measures such as lateralization: the same fronto-temporal network achieves the planning of structured linguistic expressions.

[1]  Jonathan D. Cohen,et al.  Anterior Cingulate Conflict Monitoring and Adjustments in Control , 2004, Science.

[2]  Y. Adachi,et al.  Reduction of non-periodic environmental magnetic noise in MEG measurement by continuously adjusted least squares method , 2001 .

[3]  Liina Pylkkänen,et al.  Composition of complex numbers: Delineating the computational role of the left anterior temporal lobe , 2016, NeuroImage.

[4]  David Badre,et al.  Left ventrolateral prefrontal cortex and the cognitive control of memory , 2007, Neuropsychologia.

[5]  Joachim Gross,et al.  Good practice for conducting and reporting MEG research , 2013, NeuroImage.

[6]  S. Yuval-Greenberg,et al.  The Broadband-Transient Induced Gamma-Band Response in Scalp EEG Reflects the Execution of Saccades , 2009, Brain Topography.

[7]  Peter Erhard,et al.  Involvement of classical anterior and posterior language areas in sign language production, as investigated by 4 T functional magnetic resonance imaging , 2004, Neuroscience Letters.

[8]  Murray Grossman,et al.  Causal Evidence for a Mechanism of Semantic Integration in the Angular Gyrus as Revealed by High-Definition Transcranial Direct Current Stimulation , 2016, The Journal of Neuroscience.

[9]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[10]  Adina Williams,et al.  Early sensitivity of left perisylvian cortex to relationality in nouns and verbs , 2017, Neuropsychologia.

[11]  Liina Pylkkänen,et al.  Bilingual Language Switching in the Laboratory versus in the Wild: The Spatiotemporal Dynamics of Adaptive Language Control , 2017, The Journal of Neuroscience.

[12]  Liina Pylkkänen,et al.  Adjective conjunction as a window into the LATL’s contribution to conceptual combination , 2016, Brain and Language.

[13]  Wendy Sandler,et al.  Sign Language and Linguistic Universals: Entering the lexicon: lexicalization, backformation, and cross-modal borrowing , 2006 .

[14]  Andrea E. Martin,et al.  The Anterior Midline Field: coercion or decision making? , 2009 .

[15]  P. Hagoort.  On Broca, brain, and binding: a new framework , 2005, Trends in Cognitive Sciences.

[16]  L. Pylkkänen,et al.  Scalar adjectives and the temporal unfolding of semantic composition: An MEG investigation , 2016, Neuropsychologia.

[17]  A. Braun,et al.  The neural organization of discourse: an H2 15O-PET study of narrative production in English and American sign language. , 2001, Brain : a journal of neurology.

[18]  Allen R. Braun,et al.  Language Lateralization in a Bimanual Language , 2003, Journal of Cognitive Neuroscience.

[19]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[20]  Liina Pylkkänen,et al.  An MEG Study of Silent Meaning , 2007, Journal of Cognitive Neuroscience.

[21]  Karen Emmorey,et al.  Language switching decomposed through MEG and evidence from bimodal bilinguals , 2018, Proceedings of the National Academy of Sciences.

[22]  Karen Emmorey,et al.  Neural Systems Underlying Spatial Language in American Sign Language , 2002, NeuroImage.

[23]  Liina Pylkkänen,et al.  The interplay of composition and concept specificity in the left anterior temporal lobe: An MEG study , 2015, NeuroImage.

[24]  A. Meyer.  Lexical Access in Phrase and Sentence Production: Results from Picture–Word Interference Experiments , 1996 .

[25]  Liina Pylkkänen,et al.  journal homepage: www.elsevier.com/locate/b&l Processing events: Behavioral and neuromagnetic correlates of Aspectual Coercion , 2022 .

[26]  Liina Pylkkänen,et al.  Brain bases of language selection: MEG evidence from Arabic-English bilingual language production , 2015, Front. Hum. Neurosci..

[27]  A. Caramazza,et al.  The cognate facilitation effect: implications for models of lexical access. , 2000, Journal of experimental psychology. Learning, memory, and cognition.

[28]  Ursula Bellugi,et al.  The neurobiology of sign language and its implications for the neural basis of language , 1996, Nature.

[29]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[30]  Liina Pylkkänen,et al.  Building phrases in language production: An MEG study of simple composition , 2014, Cognition.

[31]  Todd B. Parrish,et al.  Neural Correlates of Verb Argument Structure Processing , 2007, Journal of Cognitive Neuroscience.

[32]  Liina Pylkkänen,et al.  Simple Composition: A Magnetoencephalography Investigation into the Comprehension of Minimal Linguistic Phrases , 2011, The Journal of Neuroscience.

[33]  S. Dehaene,et al.  Cortical representation of the constituent structure of sentences , 2011, Proceedings of the National Academy of Sciences.

[34]  Karen Emmorey,et al.  The neural correlates of sign versus word production , 2007, NeuroImage.

[35]  U. Bellugi,et al.  The linguistic basis of left hemisphere specialization. , 1992, Science.

[36]  R. Ilmoniemi,et al.  Interpreting magnetic fields of the brain: minimum norm estimates , 2006, Medical and Biological Engineering and Computing.

[37]  J. Youd.  New framework. , 2013, Emergency nurse : the journal of the RCN Accident and Emergency Nursing Association.

[38]  Karen Emmorey,et al.  The psycholinguistics of signed and spoken languages: how biology affects processing , 2007 .

[39]  Daniel N. Osherson,et al.  Evidence for conceptual combination in the left anterior temporal lobe , 2011, NeuroImage.

[40]  J. Binder.  In defense of abstract conceptual representations , 2016, Psychonomic Bulletin & Review.

[41]  L. Pylkkänen,et al.  Bilingual Language Control in Perception versus Action: MEG Reveals Comprehension Control Mechanisms in Anterior Cingulate Cortex and Domain-General Control of Production in Dorsolateral Prefrontal Cortex , 2016, The Journal of Neuroscience.

[42]  Karen Emmorey,et al.  Neural systems underlying lexical retrieval for sign language , 2003, Neuropsychologia.

[43]  Aya Meltzer-Asscher,et al.  How the brain processes different dimensions of argument structure complexity: Evidence from fMRI , 2015, Brain and Language.

[44]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[45]  A. Braun,et al.  Neural systems for sign language production: Mechanisms supporting lexical selection, phonological encoding, and articulation , 2004, Human brain mapping.

[46]  Liina Pylkkänen,et al.  MEG evidence that the LIFG effect of object extraction requires similarity-based interference , 2013, Language and cognitive processes.

[47]  C. Fiebach,et al.  The role of left inferior frontal and superior temporal cortex in sentence comprehension: localizing syntactic and semantic processes. , 2003, Cerebral cortex.

[48]  Eric Halgren,et al.  Signed Words in the Congenitally Deaf Evoke Typical Late Lexicosemantic Responses with No Early Visual Responses in Left Superior Temporal Cortex , 2012, The Journal of Neuroscience.

[49]  Bencie Woll,et al.  Lexical and sentential processing in British Sign Language , 2006, Human brain mapping.

[50]  David Caplan,et al.  Localization of Syntactic Comprehension by Positron Emission Tomography , 1998, NeuroImage.