Motor-system dynamics during naturalistic reading of action narratives in first and second language

Do embodied semantic systems play different roles depending on when and how well a given language was learned? Emergent evidence suggests that this is the case for isolated, decontextualized stimuli, but no study has addressed the issue considering naturalistic narratives. Seeking to bridge this gap, we assessed motor-system dynamics in 26 Spanish-English bilinguals as they engaged in free, unconstrained reading of naturalistic action texts (ATs, highlighting the characters’ movements) and neutral texts (NTs, featuring low motility) in their first and second language (L1, L2). To explore functional connectivity spread over each reading session, we recorded ongoing high-density electroencephalographic signals and subjected them to functional connectivity analysis via a spatial clustering approach. Results showed that, in L1, AT (relative to NT) reading involved increased connectivity between left and right central electrodes consistently implicated in action-related processes, as well as distinct source-level modulations in motor regions. In L2, despite null group-level effects, enhanced motor-related connectivity during AT reading correlated positively with L2 proficiency and negatively with age of L2 learning. Taken together, these findings suggest that action simulations during unconstrained narrative reading involve neural couplings between motor-sensitive mechanisms, in proportion to how consolidated a language is. More generally, such evidence addresses recent calls to test the ecological validity of motor-resonance effects while offering new insights on their relation with experiential variables.

[1]  Aamir Saeed Malik,et al.  EEG Power and Functional Connectivity Correlates with Semantic Long-Term Memory Retrieval , 2018, IEEE Access.

[2]  Roel M. Willems,et al.  Embodied language in first- and second-language speakers: Neural correlates of processing motor verbs , 2014, Neuropsychologia.

[3]  M. de Vega,et al.  Understanding action language modulates oscillatory mu and beta rhythms in the same way as observing actions , 2013, Brain and Cognition.

[4]  David Huepe,et al.  Word reading and translation in bilinguals: the impact of formal and informal translation expertise , 2014, Front. Psychol..

[5]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[6]  V. Gallese,et al.  Frontal Functional Connectivity of Electrocorticographic Delta and Theta Rhythms during Action Execution Versus Action Observation in Humans , 2017, Front. Behav. Neurosci..

[7]  S. Baez,et al.  Parkinson's disease compromises the appraisal of action meanings evoked by naturalistic texts , 2017, Cortex.

[8]  Michael W. L. Chee,et al.  fMR-Adaptation and the bilingual brain , 2009, Brain and Language.

[9]  S. Dehaene,et al.  Information Sharing in the Brain Indexes Consciousness in Noncommunicative Patients , 2013, Current Biology.

[10]  Janet G. van Hell,et al.  The Revised Hierarchical Model: A critical review and assessment. , 2010, Bilingualism.

[11]  Harold Bekkering,et al.  Context Effects in Embodied Lexical-Semantic Processing , 2010, Front. Psychology.

[12]  Daniel Fraiman,et al.  Task-specific signatures in the expert brain: Differential correlates of translation and reading in professional interpreters , 2020, NeuroImage.

[13]  J. Hulstijn The construct of language proficiency in the study of bilingualism from a cognitive perspective* , 2012, Bilingualism: Language and Cognition.

[14]  Michel Paradis,et al.  Declarative and Procedural Determinants of Second Languages , 2009 .

[15]  Fan Zhang,et al.  Language history questionnaire (LHQ 2.0): A new dynamic web-based research tool , 2014 .

[16]  G. Buccino,et al.  Fluent Speakers of a Second Language Process Graspable Nouns Expressed in L2 Like in Their Native Language , 2017, Front. Psychol..

[17]  Manuel de Vega,et al.  Brain dynamics in the comprehension of action-related language. A time-frequency analysis of mu rhythms , 2015, NeuroImage.

[18]  Barbara Kaup,et al.  Embodiment and second-language: Automatic activation of motor responses during processing spatially associated L2 words and emotion L2 words in a vertical Stroop paradigm , 2014, Brain and Language.

[19]  Y. Shtyrov,et al.  Automatic ultrarapid activation and inhibition of cortical motor systems in spoken word comprehension , 2014, Proceedings of the National Academy of Sciences.

[20]  Agustín Ibáñez,et al.  Cortical dynamics and subcortical signatures of motor-language coupling in Parkinson’s disease , 2015, Scientific Reports.

[21]  Klaus Gramann,et al.  Spatial Cognition & Computation: an Interdisciplinary Journal Embodiment of Spatial Reference Frames and Individual Differences in Reference Frame Proclivity Embodiment of Spatial Reference Frames and Individual Differences in Reference Frame Proclivity , 2022 .

[22]  F. Pulvermüller How neurons make meaning: brain mechanisms for embodied and abstract-symbolic semantics , 2013, Trends in Cognitive Sciences.

[23]  Adolfo M García,et al.  How meaning unfolds in neural time: Embodied reactivations can precede multimodal semantic effects during language processing , 2019, NeuroImage.

[24]  Sammi R. Chekroud,et al.  Concurrent visual and motor selection during visual working memory guided action , 2018, Nature Neuroscience.

[25]  P. Gordon,et al.  Reading words in discourse: the modulation of lexical priming effects by message-level context. , 2006, Behavioral and cognitive neuroscience reviews.

[26]  Nikola Vukovic,et al.  When words get physical: evidence for proficiency-modulated somatotopic motor interference during second language comprehension , 2013, CogSci.

[27]  P. Rossini,et al.  Alpha, beta and gamma electrocorticographic rhythms in somatosensory, motor, premotor and prefrontal cortical areas differ in movement execution and observation in humans , 2016, Clinical Neurophysiology.

[28]  John M. Henderson,et al.  Toward Semantics in the Wild: Activation to Manipulable Nouns in Naturalistic Reading , 2016, The Journal of Neuroscience.

[29]  Francisco Szigriszt Pazos Sistemas predictivos de legilibilidad del mensaje escrito: fórmula de perspicuidad , 2012 .

[30]  Timothy H Lucas,et al.  Functional separation of languages in the bilingual brain: a comparison of electrical stimulation language mapping in 25 bilingual patients and 117 monolingual control patients. , 2004, Journal of neurosurgery.

[31]  Margarita Kaushanskaya,et al.  The Language Experience and Proficiency Questionnaire (LEAP-Q): assessing language profiles in bilinguals and multilinguals. , 2007, Journal of speech, language, and hearing research : JSLHR.

[32]  Clemens Brunner,et al.  Event-related EEG theta and alpha band oscillatory responses during language translation , 2007, Brain Research Bulletin.

[33]  Diana Stojanovic,et al.  Body part representations in verbal semantics , 2010, Memory & cognition.

[34]  Nikola Vukovic,et al.  Cortical motor systems are involved in second-language comprehension: Evidence from rapid mu-rhythm desynchronisation , 2014, NeuroImage.

[35]  Roel M. Willems,et al.  Body-specific representations of action verbs: Neural evidence from right- and left-handers , 2009, NeuroImage.

[36]  Robert J. Zatorre,et al.  Bilingual brain organization: A functional magnetic resonance adaptation study , 2006, NeuroImage.

[37]  J. Kroll,et al.  FIRST LANGUAGE ACTIVATION DURING SECOND LANGUAGE LEXICAL PROCESSING: An Investigation of Lexical Form, Meaning, and Grammatical Class , 2006, Studies in Second Language Acquisition.

[38]  Emiliano Ricciardi,et al.  EEG functional connectivity metrics wPLI and wSMI account for distinct types of brain functional interactions , 2018, Scientific Reports.

[39]  Fernando Marmolejo-Ramos,et al.  The linguistic context effects on the processing of body–object interaction words: An ERP study on second language learners , 2015, Brain Research.

[40]  Aldo Quattrone,et al.  Viewing photos and reading nouns of natural graspable objects similarly modulate motor responses , 2014, Front. Hum. Neurosci..

[41]  Arthur M Glenberg,et al.  Moved by Reading in a Spanish-Speaking, Dual Language Learner Population. , 2018, Language, speech, and hearing services in schools.

[42]  Margot J. Taylor,et al.  Guidelines for using human event-related potentials to study cognition: recording standards and publication criteria. , 2000, Psychophysiology.

[43]  F. Pulvermüller Neural reuse of action perception circuits for language, concepts and communication , 2018, Progress in Neurobiology.

[44]  Laura Sabourin,et al.  The effect of age of L2 acquisition on the organization of the bilingual lexicon: evidence from masked priming , 2014 .

[45]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[46]  Stefan Elmer,et al.  Functional Connectivity in the Left Dorsal Stream Facilitates Simultaneous Language Translation: An EEG Study , 2016, Front. Hum. Neurosci..

[47]  Ping Li,et al.  Language History Questionnaire (LHQ3): An enhanced tool for assessing multilingual experience , 2019, Bilingualism: Language and Cognition.

[48]  M. Sigman,et al.  Attention, in and Out: Scalp-Level and Intracranial EEG Correlates of Interoception and Exteroception , 2017, Front. Neurosci..

[49]  D. Perani,et al.  Neural convergence for language comprehension and grammatical class production in highly proficient bilinguals is independent of age of acquisition , 2013, Cortex.

[50]  Peter Rappelsberger,et al.  Characteristic functional networks in high- versus low-proficiency second language speakers detected also during native language processing: an explorative EEG coherence study in 6 frequency bands. , 2005, Brain research. Cognitive brain research.

[51]  Boris Gourévitch,et al.  Evaluating information transfer between auditory cortical neurons. , 2007, Journal of neurophysiology.

[52]  Raffaella Ida Rumiati,et al.  The role of volleyball expertise in motor simulation. , 2012, Acta psychologica.

[53]  Ajay S. Pillai,et al.  Decreased Modulation of EEG Oscillations in High-Functioning Autism during a Motor Control Task , 2016, Front. Hum. Neurosci..

[54]  A. Ibanez,et al.  Motor-language coupling: Direct evidence from early Parkinson’s disease and intracranial cortical recordings , 2013, Cortex.

[55]  Fabricio Baglivo,et al.  Consistent Gradient of Performance and Decoding of Stimulus Type and Valence From Local and Network Activity , 2019, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[56]  J. Kroll,et al.  The neurocognition of recovery patterns in bilingual aphasics , 2004 .

[57]  H. Langdon,et al.  Dual-Dimension Naming Speed and Language-Dominance Ratings by Bilingual Hispanic Adults , 2005 .

[58]  Marco Tettamanti,et al.  Training of Manual Actions Improves Language Understanding of Semantically Related Action Sentences , 2012, Front. Psychology.

[59]  Richard M. Leahy,et al.  Brainstorm: A User-Friendly Application for MEG/EEG Analysis , 2011, Comput. Intell. Neurosci..

[60]  Agustín Ibáñez,et al.  The cerebellum and embodied semantics: evidence from a case of genetic ataxia due to STUB1 mutations , 2016, Journal of Medical Genetics.

[61]  Roel M. Willems,et al.  Complementary Systems for Understanding Action Intentions , 2008, Current Biology.

[62]  Lorraine K. Tyler,et al.  Objects and their actions: evidence for a neurally distributed semantic system , 2003, NeuroImage.

[63]  Agustín Ibáñez,et al.  The Embodied Penman: Effector-Specific Motor-Language Integration During Handwriting , 2019, Cogn. Sci..

[64]  F. Manes,et al.  Applauding with Closed Hands: Neural Signature of Action-Sentence Compatibility Effects , 2010, PloS one.

[65]  G. Pfurtscheller,et al.  Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. , 2005, Brain research. Cognitive brain research.

[66]  Fan Cao,et al.  L1 and L2 processing in the bilingual brain: A meta-analysis of neuroimaging studies , 2016, Brain and Language.

[67]  T. Dijkstra,et al.  The architecture of the bilingual word recognition system: From identification to decision , 2002, Bilingualism: Language and Cognition.

[68]  Ulrika Kahl,et al.  [The bilingual brain]. , 2002, Lakartidningen.

[69]  Agustín Ibáñez,et al.  The Road Less Traveled: Alternative Pathways for Action-Verb Processing in Parkinson's Disease. , 2016, Journal of Alzheimer's disease : JAD.

[70]  R D Pascual-Marqui,et al.  Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. , 2002, Methods and findings in experimental and clinical pharmacology.

[71]  Michael T. Ullman,et al.  The neural basis of lexicon and grammar in first and second language: the declarative/procedural model , 2001 .

[72]  C. Davis,et al.  BuscaPalabras: A program for deriving orthographic and phonological neighborhood statistics and other psycholinguistic indices in Spanish , 2005, Behavior research methods.

[73]  G. Rizzolatti,et al.  Congruent Embodied Representations for Visually Presented Actions and Linguistic Phrases Describing Actions , 2006, Current Biology.

[74]  A. Ibáñez,et al.  Losing ground: Frontostriatal atrophy disrupts language embodiment in Parkinson’s and Huntington’s disease , 2017, Neuroscience & Biobehavioral Reviews.

[75]  Roel M. Willems,et al.  Grounding the neurobiology of language in first principles: The necessity of non-language-centric explanations for language comprehension , 2018, Cognition.

[76]  Daniel Fraiman,et al.  Towards affordable biomarkers of frontotemporal dementia: A classification study via network’s information sharing , 2017, Scientific Reports.

[77]  O. Sporns,et al.  From regions to connections and networks: new bridges between brain and behavior , 2016, Current Opinion in Neurobiology.

[78]  S. Baez,et al.  Feeling, learning from and being aware of inner states: interoceptive dimensions in neurodegeneration and stroke , 2016, Philosophical Transactions of the Royal Society B: Biological Sciences.

[79]  B. Kaup,et al.  Grounded cognition: Comparing Language × Space interactions in first language and second language , 2017, Applied Psycholinguistics.

[80]  F. Pulvermüller,et al.  Movement priming of EEG/MEG brain responses for action-words characterizes the link between language and action , 2016, Cortex.

[81]  Agustín Ibáñez,et al.  A touch with words: Dynamic synergies between manual actions and language , 2016, Neuroscience & Biobehavioral Reviews.

[82]  Fabricio Baglivo,et al.  Early detection of intentional harm in the human amygdala. , 2016, Brain : a journal of neurology.

[83]  Jubin Abutalebi,et al.  Neural aspects of second language representation and language control. , 2008, Acta psychologica.

[84]  F. Pulvermüller Semantic embodiment, disembodiment or misembodiment? In search of meaning in modules and neuron circuits , 2013, Brain and Language.

[85]  A. Pavlenko,et al.  Affective processing in bilingual speakers: disembodied cognition? , 2012, International journal of psychology : Journal international de psychologie.

[86]  Adolfo M. García Translating with an Injured Brain: Neurolinguistic Aspects of Translation as Revealed by Bilinguals with Cerebral Lesions , 2015 .

[87]  Friedemann Pulvermüller,et al.  When do you grasp the idea? MEG evidence for instantaneous idiom understanding , 2012, NeuroImage.

[88]  Martina G. Vilas,et al.  Bilingual memory, to the extreme: Lexical processing in simultaneous interpreters , 2018, Bilingualism: Language and Cognition.

[89]  Michael Halliday,et al.  An Introduction to Functional Grammar , 1985 .

[90]  F. Manes,et al.  The inner world of overactive monitoring: neural markers of interoception in obsessive–compulsive disorder , 2017, Psychological Medicine.

[91]  A. Ibáñez,et al.  Too late to be grounded? Motor resonance for action words acquired after middle childhood , 2019, Brain and Cognition.

[92]  Rolf A. Zwaan,et al.  Seeing, acting, understanding: motor resonance in language comprehension. , 2006, Journal of experimental psychology. General.

[93]  T. Gollan,et al.  Self-ratings of spoken language dominance: A Multilingual Naming Test (MINT) and preliminary norms for young and aging Spanish–English bilinguals* , 2011, Bilingualism: Language and Cognition.

[94]  I. M. Barrio-Cantalejo,et al.  Validación de la Escala INFLESZ para evaluar la legibilidad de los textos dirigidos a pacientes , 2008 .

[95]  Arno Klein,et al.  Mindboggle: a scatterbrained approach to automate brain labeling , 2005, NeuroImage.

[96]  Piergiorgio Trevisan,et al.  Systemic functional grammar as a tool for experimental stimulus design: new appliable horizons in psycholinguistics and neurolinguistics , 2019, Language Sciences.

[97]  Katie Wagner,et al.  Carpet or Cárcel: The effect of age of acquisition and language mode on bilingual lexical access , 2010 .

[98]  Giuseppe Scialfa,et al.  Nouns and verbs in the brain: Grammatical class and task specific effects as revealed by fMRI , 2008, Cognitive neuropsychology.

[99]  Kenneth Hyltenstam,et al.  Revisiting the bilingual lexical deficit: The impact of age of acquisition , 2019, Cognition.

[100]  V. Gallese,et al.  The neural exploitation hypothesis and its implications for an embodied approach to language and cognition: Insights from the study of action verbs processing and motor disorders in Parkinson's disease , 2018, Cortex.

[101]  Arthur M. Jacobs,et al.  Can Harry Potter still put a spell on us in a second language? An fMRI study on reading emotion-laden literature in late bilinguals , 2015, Cortex.

[102]  Luis Montesano,et al.  Evolution of EEG Motor Rhythms after Spinal Cord Injury: A Longitudinal Study , 2015, PloS one.

[103]  Bin He,et al.  EEG Source Imaging Enhances the Decoding of Complex Right-Hand Motor Imagery Tasks , 2016, IEEE Transactions on Biomedical Engineering.

[104]  Erin Walker,et al.  When (and How) Interacting With Technology-Enhanced Storybooks Helps Dual Language Learners , 2017 .

[105]  L. Tyler,et al.  Modulation of motor and premotor cortices by actions, action words and action sentences , 2009, Neuropsychologia.

[106]  G. Pfurtscheller,et al.  Motor imagery activates primary sensorimotor area in humans , 1997, Neuroscience Letters.

[107]  V. Gallese,et al.  What is so special about embodied simulation? , 2011, Trends in Cognitive Sciences.

[108]  Angela D. Friederici,et al.  Comprehending Prehending: Neural Correlates of Processing Verbs with Motor Stems , 2007, Journal of Cognitive Neuroscience.

[109]  T. Oh,et al.  Age and Proficiency in the Bilingual Brain Revisited: Activation Patterns Across Different L2-Learner Types , 2019, Front. Commun..

[110]  Mia Liljeström,et al.  Perceiving and naming actions and objects , 2008, NeuroImage.

[111]  Wolfgang Klimesch,et al.  Long‐Term Effects of Gestures on Memory for Foreign Language Words Trained in the Classroom , 2014 .

[112]  A. Greco,et al.  Language use affects proficiency in Italian–Spanish bilinguals irrespective of age of second language acquisition* , 2014, Bilingualism: Language and Cognition.

[113]  Sian L. Beilock,et al.  Sports experience changes the neural processing of action language , 2008, Proceedings of the National Academy of Sciences.

[114]  Marta Maieron,et al.  How are the motor system activity and functional connectivity between the cognitive and sensorimotor systems modulated by athletic expertise? , 2013, Brain Research.

[115]  Natalia Trujillo,et al.  Gesture influences the processing of figurative language in non-native speakers: ERP evidence , 2010, Neuroscience Letters.

[116]  C. Davis N-Watch: A program for deriving neighborhood size and other psycholinguistic statistics , 2005, Behavior research methods.

[117]  Théodore Papadopoulo,et al.  OpenMEEG: opensource software for quasistatic bioelectromagnetics , 2010, Biomedical engineering online.

[118]  G. Tononi,et al.  A Theoretically Based Index of Consciousness Independent of Sensory Processing and Behavior , 2013, Science Translational Medicine.

[119]  Gert Cauwenberghs,et al.  EEG channel interpolation using ellipsoid geodesic length , 2016, 2016 IEEE Biomedical Circuits and Systems Conference (BioCAS).

[120]  J. Annoni,et al.  Embodied Semantics in a Second Language: Critical Review and Clinical Implications , 2019, Front. Hum. Neurosci..

[121]  A. Caramazza,et al.  Concepts Are More than Percepts: The Case of Action Verbs , 2008, The Journal of Neuroscience.

[122]  José E. García-Albea,et al.  Translation performance of beginning, intermediate and proficient Spanish-Catalan bilinguals: Effects of form and semantic relations , 2008 .

[123]  Miguel Martorell Caro,et al.  Reading Shakespearean tropes in a foreign tongue: Age of L2 acquisition modulates neural responses to functional shifts , 2019, Neuropsychologia.

[124]  Francesco Foroni,et al.  Do we embody second language? Evidence for ‘partial’ simulation during processing of a second language , 2015, Brain and Cognition.

[125]  I. Johnsrude,et al.  Somatotopic Representation of Action Words in Human Motor and Premotor Cortex , 2004, Neuron.

[126]  Agustín Ibáñez,et al.  A moving story: Whole-body motor training selectively improves the appraisal of action meanings in naturalistic narratives , 2017, Scientific Reports.

[127]  G. Riva,et al.  Is Motor Simulation Involved During Foreign Language Learning? A Virtual Reality Experiment , 2015 .

[128]  Ping Li,et al.  Language history questionnaire: A Web-based interface for bilingual research , 2006, Behavior research methods.

[129]  George A. Mashour,et al.  Assessing levels of consciousness with symbolic analysis , 2015, Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences.

[130]  A. Ibáñez,et al.  Hands typing what hands do: Action–semantic integration dynamics throughout written verb production , 2016, Cognition.

[131]  T. Dijkstra,et al.  Multilink: a computational model for bilingual word recognition and word translation , 2018, Bilingualism: Language and Cognition.

[132]  Clemens Brunner,et al.  Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks , 2006, NeuroImage.

[133]  V. Ferreira,et al.  When a seven is not a seven: Self-ratings of bilingual language proficiency differ between and within language populations , 2018, Bilingualism: Language and Cognition.

[134]  Yasuhiro Haruta,et al.  Lateralized Theta Wave Connectivity and Language Performance in 2- to 5-Year-Old Children , 2011, The Journal of Neuroscience.

[135]  Guido Nolte,et al.  Brain Oscillations and Functional Connectivity during Overt Language Production , 2012, Front. Psychology.