Neural representation of words within phrases: Temporal evolution of color-adjectives and object-nouns during simple composition

In language, stored representations of lexical items combine into an infinitude of complex expressions. While the neuroscience of composition has begun to mature, we do not yet understand how the stored representations evolve and morph during composition. New decoding techniques allow us to crack open this very hard question: we can train a model to recognize a representation in one context or time-point and assess its accuracy in another. We combined the decoding approach with magnetoencephalography recorded during a picture naming task to investigate the temporal evolution of noun and adjective representations during speech planning. We tracked word representations as they combined into simple two-word phrases, using single words and two-word lists as non-combinatory controls. We found that nouns were generally more decodable than adjectives, suggesting that noun representations were stronger and/or more consistent across trials than those of adjectives. When training and testing across contexts and times, the representations of isolated nouns were recoverable when those nouns were embedded in phrases, but not so if they were embedded in lists. Adjective representations did not show a similar consistency across isolated and phrasal contexts. Noun representations in phrases also sustained over time in a way that was not observed for any other pairing of word class and context. These findings offer a new window into the temporal evolution and context sensitivity of word representations during composition, revealing a clear asymmetry between adjectives and nouns. The impact of phrasal contexts on the decodability of nouns may be due to the nouns’ status as head of phrase—an intriguing hypothesis for future research.

[1]  Tom Mitchell,et al.  Relating Simple Sentence Representations in Deep Neural Networks and the Brain , 2019, ACL.

[2]  Liina Pylkkänen,et al.  The neural basis of combinatory syntax and semantics , 2019, Science.

[3]  Benjamin Cohen,et al.  Models of Concepts , 1984, Cogn. Sci..

[4]  Liina Pylkkänen,et al.  Building phrases in language production: An MEG study of simple composition , 2014, Cognition.

[5]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[6]  V. Ferreira,et al.  The priming of basic combinatory responses in MEG , 2018, Cognition.

[7]  L. Bruzzone,et al.  EEG decoding of semantic category reveals distributed representations for single concepts , 2011, Brain and Language.

[8]  Thomas L. Griffiths,et al.  Supplementary Information for Natural Speech Reveals the Semantic Maps That Tile Human Cerebral Cortex , 2022 .

[9]  J. Tiihonen,et al.  Amygdala-orbitofrontal structural and functional connectivity in females with anxiety disorders, with and without a history of conduct disorder , 2018, Scientific Reports.

[10]  Alona Fyshe,et al.  The lexical semantics of adjective–noun phrases in the human brain , 2019, Human brain mapping.

[11]  Karen Emmorey,et al.  Shared neural correlates for building phrases in signed and spoken language , 2018, Scientific Reports.

[12]  Katrin Erk,et al.  Vector Space Models of Word Meaning and Phrase Meaning: A Survey , 2012, Lang. Linguistics Compass.

[13]  Y. Benjamini,et al.  THE CONTROL OF THE FALSE DISCOVERY RATE IN MULTIPLE TESTING UNDER DEPENDENCY , 2001 .

[14]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[15]  Tom Michael Mitchell,et al.  Predicting Human Brain Activity Associated with the Meanings of Nouns , 2008, Science.

[16]  Riitta Salmelin,et al.  Tracking neural coding of perceptual and semantic features of concrete nouns , 2012, NeuroImage.

[17]  Alona Fyshe,et al.  Studying language in context using the temporal generalization method , 2019, Philosophical Transactions of the Royal Society B.

[18]  Alexander G. Huth,et al.  Incorporating Context into Language Encoding Models for fMRI , 2018, bioRxiv.

[19]  W J Levelt,et al.  Spoken word production: A theory of lexical access , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[20]  F. Pulvermüller,et al.  Early Parallel Activation of Semantics and Phonology in Picture Naming: Evidence from a Multiple Linear Regression MEG Study , 2014, Cerebral cortex.

[21]  Louise McNally,et al.  Color, context, and compositionality , 2010, Synthese.

[22]  F. De Filippis,et al.  A Selected Core Microbiome Drives the Early Stages of Three Popular Italian Cheese Manufactures , 2014, PloS one.

[23]  S. Dehaene,et al.  Characterizing the dynamics of mental representations: the temporal generalization method , 2014, Trends in Cognitive Sciences.

[24]  Giulia V. Elli,et al.  Knowledge of animal appearance among sighted and blind adults , 2019, Proceedings of the National Academy of Sciences.

[25]  Brian Murphy,et al.  Simultaneously Uncovering the Patterns of Brain Regions Involved in Different Story Reading Subprocesses , 2014, PloS one.