Differentiating Semantic Categories during the Acquisition of Novel Words: Correspondence Analysis Applied to Event-related Potentials

Growing evidence suggests that semantic knowledge is represented in distributed neural networks that include modality-specific structures. Here, we examined the processes underlying the acquisition of words from different semantic categories to determine whether the emergence of visual- and action-based categories could be tracked back to their acquisition. For this, we applied correspondence analysis (CA) to ERPs recorded at various moments during acquisition. CA is a multivariate statistical technique typically used to reveal distance relationships between words of a corpus. Applied to ERPs, it allows isolating factors that best explain variations in the data across time and electrodes. Participants were asked to learn new action and visual words by associating novel pseudowords with the execution of hand movements or the observation of visual images. Words were probed before and after training on two consecutive days. To capture processes that unfold during lexical access, CA was applied on the 100–400 msec post-word onset interval. CA isolated two factors that organized the data as a function of test sessions and word categories. Conventional ERP analyses further revealed a category-specific increase in the negativity of the ERPs to action and visual words at the frontal and occipital electrodes, respectively. The distinct neural processes underlying action and visual words can thus be tracked back to the acquisition of word-referent relationships and may have its origin in association learning. Given current evidence for the flexibility of language-induced sensory-motor activity, we argue that these associative links may serve functions beyond word understanding, that is, the elaboration of situation models.

[1]  Thomas Wolbers,et al.  Neural Dynamics of Learning Sound—Action Associations , 2008, PloS one.

[2]  Friedemann Pulvermüller,et al.  Rapid Cortical Plasticity Underlying Novel Word Learning , 2010, The Journal of Neuroscience.

[3]  Leslie G. Ungerleider,et al.  Neural correlates of category-specific knowledge , 1996, Nature.

[4]  Daniel Casasanto,et al.  Flexibility in Embodied Language Understanding , 2011, Front. Psychology.

[5]  I. Johnsrude,et al.  Somatotopic Representation of Action Words in Human Motor and Premotor Cortex , 2004, Neuron.

[6]  Stefan Knecht,et al.  New Names for Known Things: On the Association of Novel Word Forms with Existing Semantic Information , 2010, Journal of Cognitive Neuroscience.

[7]  F. Pulvermüller,et al.  Category-specific conceptual processing of color and form in left fronto-temporal cortex. , 2006, Cerebral cortex.

[8]  Werner Lutzenberger,et al.  Evoked potentials distinguish between nouns and verbs , 1995, Neuroscience Letters.

[9]  G. Rizzolatti,et al.  Congruent Embodied Representations for Visually Presented Actions and Linguistic Phrases Describing Actions , 2006, Current Biology.

[10]  M. Gaskell,et al.  Lexical competition and the acquisition of novel words , 2003, Cognition.

[11]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[12]  Friedemann Pulvermüller,et al.  Ultra-rapid access to words in the brain , 2012, Nature Communications.

[13]  Sydney S. Cash,et al.  Decoding word and category-specific spatiotemporal representations from MEG and EEG , 2011, NeuroImage.

[14]  Gereon R. Fink,et al.  Action verbs and the primary motor cortex: A comparative TMS study of silent reading, frequency judgments, and motor imagery , 2008, Neuropsychologia.

[15]  P. Hagoort,et al.  Language beyond action , 2008, Journal of Physiology-Paris.

[16]  M. Kiefer,et al.  Conceptual representations in mind and brain: Theoretical developments, current evidence and future directions , 2012, Cortex.

[17]  Michael J Cortese,et al.  Handbook of Psycholinguistics , 2011 .

[18]  Stéphanie Dabic,et al.  Toward a neurolexicology: A method for exploring the organization of the mental lexicon by analyzing electrophysiological signals , 2012 .

[19]  Friedemann Pulverm Uuml,et al.  Words in the brain's language , 1999 .

[20]  L. Otten,et al.  Event-related potentials to event-related words: Grammatical class and semantic attributes in the representation of knowledge , 2010, Brain Research.

[21]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[22]  Vittorio Gallese,et al.  Listening to Action-related Sentences Activates Fronto-parietal Motor Circuits , 2005, Journal of Cognitive Neuroscience.

[23]  H. Gastaut [Electrocorticographic study of the reactivity of rolandic rhythm]. , 1952, Revue neurologique.

[24]  G. Lakoff,et al.  The Brain's concepts: the role of the Sensory-motor system in conceptual knowledge , 2005, Cognitive neuropsychology.

[25]  Lenhart K. Schubert Semantic Representation , 2015, AAAI.

[26]  M. Gaskell,et al.  Sleep-Associated Changes in the Mental Representation of Spoken Words , 2007, Psychological science.

[27]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[28]  Rolf A. Zwaan,et al.  Motor Resonance and Linguistic Focus , 2008, Quarterly journal of experimental psychology.

[29]  E. Saltzman,et al.  Action Representation of Sound: Audiomotor Recognition Network While Listening to Newly Acquired Actions , 2007, The Journal of Neuroscience.

[30]  Bradford Z. Mahon,et al.  A critical look at the embodied cognition hypothesis and a new proposal for grounding conceptual content , 2008, Journal of Physiology-Paris.

[31]  Leslie G. Ungerleider,et al.  Discrete Cortical Regions Associated with Knowledge of Color and Knowledge of Action , 1995, Science.

[32]  W. K. Simmons,et al.  A common neural substrate for perceiving and knowing about color , 2007, Neuropsychologia.

[33]  L. Barsalou Grounded cognition. , 2008, Annual review of psychology.

[34]  Markus Kiefer,et al.  Conceptual Flexibility in the Human Brain: Dynamic Recruitment of Semantic Maps from Visual, Motor, and Motion-related Areas , 2008, Journal of Cognitive Neuroscience.

[35]  F. Pulvermüller,et al.  Distributed neuronal networks for encoding category‐specific semantic information: the mismatch negativity to action words , 2004, The European journal of neuroscience.

[36]  F. Pulvermüller,et al.  The time course of action and action-word comprehension in the human brain as revealed by neurophysiology , 2008, Journal of Physiology-Paris.

[37]  Bernard Victorri,et al.  Construction d'espaces sémantiques à l'aide de dictionnaires de synonymes , 1998 .

[38]  Tom Michael Mitchell,et al.  Predicting Human Brain Activity Associated with the Meanings of Nouns , 2008, Science.

[39]  Viviane Deprez,et al.  Action relevance in linguistic context drives word-induced motor activity , 2014, Front. Hum. Neurosci..

[40]  Ralph-Axel Müller,et al.  Category-specific activations during word generation reflect experiential sensorimotor modalities , 2009, NeuroImage.

[41]  J. Karhu,et al.  Repetition suppression in the cortical motor and auditory systems resemble each other – A combined TMS and evoked potential study , 2013, Neuroscience.

[42]  Stierlin Organization of Behavior. A Neuropsychological Theory , 1953 .

[43]  Yury Shtyrov,et al.  Fast Mapping of Novel Word Forms Traced Neurophysiologically , 2011, Front. Psychology.

[44]  Mark S. Seidenberg,et al.  Activation of sensory-motor areas in sentence comprehension. , 2010, Cerebral cortex.

[45]  Dietrich Lehmann,et al.  Microstates in Language-Related Brain Potential Maps Show Noun–Verb Differences , 1996, Brain and Language.

[46]  A. Rodríguez-Fornells,et al.  Watching the brain during meaning acquisition. , 2007, Cerebral cortex.

[47]  E. Capaldi,et al.  The organization of behavior. , 1992, Journal of applied behavior analysis.

[48]  V. Gallese,et al.  What is so special about embodied simulation? , 2011, Trends in Cognitive Sciences.

[49]  O. Hauk,et al.  Neurophysiological distinction of action words in the fronto‐central cortex , 2004, Human brain mapping.

[50]  Y. Paulignan,et al.  Learning to associate novel words with motor actions: Language-induced motor activity following short training , 2012, Cortex.

[51]  M. Kiefer,et al.  The Sound of Concepts: Four Markers for a Link between Auditory and Conceptual Brain Systems , 2008, The Journal of Neuroscience.

[52]  Friedemann Pulvermüller,et al.  Category specificity in the processing of color-related and form-related words: An ERP study , 2006, NeuroImage.

[53]  Friedemann Pulvermüller,et al.  Reading cinnamon activates olfactory brain regions , 2006, NeuroImage.

[54]  Friedemann Pulvermüller,et al.  Brain mechanisms linking language and action , 2005, Nature Reviews Neuroscience.

[55]  Friedemann Pulvermüller,et al.  Brain Signatures of Meaning Access in Action Word Recognition , 2005, Journal of Cognitive Neuroscience.

[56]  Rutvik H. Desai,et al.  The neurobiology of semantic memory , 2011, Trends in Cognitive Sciences.

[57]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[58]  F Pulvermüller,et al.  Nouns and verbs in the intact brain: evidence from event-related potentials and high-frequency cortical responses. , 1999, Cerebral cortex.

[59]  F. Pulvermüller,et al.  Walking or Talking?: Behavioral and Neurophysiological Correlates of Action Verb Processing , 2001, Brain and Language.

[60]  B. Mesquita,et al.  Adjustment to Chronic Diseases and Terminal Illness Health Psychology : Psychological Adjustment to Chronic Disease , 2006 .

[61]  A. Damasio,et al.  A neural basis for the retrieval of conceptual knowledge , 1997, Neuropsychologia.

[62]  Gereon R. Fink,et al.  Stimulus properties matter more than perspective: An fMRI study of mental imagery and silent reading of action phrases , 2007, NeuroImage.

[63]  S. Cappa,et al.  Neural representations of unfamiliar objects are modulated by sensorimotor experience , 2013, Cortex.

[64]  Marcel Brass,et al.  Through the looking glass: counter‐mirror activation following incompatible sensorimotor learning , 2008, The European journal of neuroscience.

[65]  Jonas Larsson,et al.  fMRI repetition suppression: neuronal adaptation or stimulus expectation? , 2012, Cerebral cortex.

[66]  A. Friederici Towards a neural basis of auditory sentence processing , 2002, Trends in Cognitive Sciences.

[67]  B A Goldfield,et al.  Nouns before verbs in comprehension vs. production: the view from pragmatics , 2000, Journal of Child Language.

[68]  Michael K. Tanenhaus,et al.  Neural correlates of partial lexical activation , 2008, Proceedings of the National Academy of Sciences.

[69]  Friedemann Pulvermüller,et al.  Meaning and the brain: The neurosemantics of referential, interactive, and combinatorial knowledge , 2012, Journal of Neurolinguistics.

[70]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[71]  Rolf A. Zwaan,et al.  Discourse comprehension. , 1997, Annual review of psychology.

[72]  P. Johnson-Laird,et al.  Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness , 1985 .

[73]  Alex Martin,et al.  Grounding Object Concepts in Perception and Action: Evidence from FMRI Studies of Tools , 2007, Cortex.

[74]  Viviane Deprez,et al.  Grip Force Reveals the Context Sensitivity of Language-Induced Motor Activity during “Action Words” Processing: Evidence from Sentential Negation , 2012, PloS one.

[75]  N. Birbaumer,et al.  Brain Rhythms of Language: Nouns Versus Verbs , 1996, The European journal of neuroscience.

[76]  Friedemann Pulvermüller,et al.  When do you grasp the idea? MEG evidence for instantaneous idiom understanding , 2012, NeuroImage.

[77]  F. Pulvermüller,et al.  Grasping Ideas with the Motor System: Semantic Somatotopy in Idiom Comprehension , 2009 .

[78]  S. Cappa,et al.  Nouns and verbs in the brain: A review of behavioural, electrophysiological, neuropsychological and imaging studies , 2011, Neuroscience & Biobehavioral Reviews.

[79]  Matthew H. Davis,et al.  A complementary systems account of word learning: neural and behavioural evidence , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[80]  Antoni Rodríguez-Fornells,et al.  Neural Differences in the Mapping of Verb and Noun Concepts onto Novel Words , 2022 .

[81]  F. Pulvermüller Brain reflections of words and their meaning , 2001, Trends in Cognitive Sciences.

[82]  O Bertrand,et al.  A theoretical justification of the average reference in topographic evoked potential studies. , 1985, Electroencephalography and clinical neurophysiology.

[83]  F. Pulvermüller,et al.  Words in the brain's language , 1999, Behavioral and Brain Sciences.

[84]  F. Pulvermüller,et al.  Reading salt activates gustatory brain regions: fMRI evidence for semantic grounding in a novel sensory modality. , 2012, Cerebral cortex.

[85]  Alfonso Caramazza,et al.  The orchestration of the sensory-motor systems: Clues from Neuropsychology , 2005, Cognitive neuropsychology.

[86]  Alex Martin,et al.  Semantic memory and the brain: structure and processes , 2001, Current Opinion in Neurobiology.

[87]  W. Schneider,et al.  Perceptual Knowledge Retrieval Activates Sensory Brain Regions , 2006, The Journal of Neuroscience.

[88]  F. Pulvermüller,et al.  Semantic or lexico-syntactic factors: what determines word-class specific activity in the human brain? , 1999, Neuroscience Letters.

[89]  T. Talavage,et al.  Neuroanatomical distribution of five semantic components of verbs: Evidence from fMRI , 2008, Brain and Language.

[90]  Friedemann Pulvermüller,et al.  Sensorimotor semantics on the spot: brain activity dissociates between conceptual categories within 150 ms , 2013, Scientific Reports.

[91]  Marina Laganaro,et al.  Neural dynamics of object noun, action verb and action noun production in picture naming , 2015, Brain and Language.

[92]  Alex Martin,et al.  A neural system for learning about object function. , 2006, Cerebral cortex.

[93]  William D. Marslen-Wilson,et al.  The time course of visual word recognition as revealed by linear regression analysis of ERP data , 2006, NeuroImage.

[94]  Caroline Catmur,et al.  Sensorimotor Learning Configures the Human Mirror System , 2007, Current Biology.