Modeling Affirmative and Negated Action Processing in the Brain with Lexical and Compositional Semantic Models

Recent work shows that distributional semantic models can be used to decode patterns of brain activity associated with individual words and sentence meanings. However, it is yet unclear to what extent such models can be used to study and decode fMRI patterns associated with specific aspects of semantic composition such as the negation function. In this paper, we apply lexical and compositional semantic models to decode fMRI patterns associated with negated and affirmative sentences containing hand-action verbs. Our results show reduced decoding (correlation) of sentences where the verb is in the negated context, as compared to the affirmative one, within brain regions implicated in action-semantic processing. This supports behavioral and brain imaging studies, suggesting that negation involves reduced access to aspects of the affirmative mental representation. The results pave the way for testing alternate semantic models of negation against human semantic processing in the brain.

[1]  Stephen Clark,et al.  Speaking, Seeing, Understanding: Correlating semantic models with conceptual representation in the brain , 2017, EMNLP.

[2]  Amy Perfors,et al.  Predicting human similarity judgments with distributional models: The value of word associations. , 2016, COLING.

[3]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[4]  Lorella Battelli,et al.  The Default Computation of Negated Meanings , 2016, Journal of Cognitive Neuroscience.

[5]  Friedemann Pulvermüller,et al.  Representational Similarity Mapping of Distributional Semantics in Left Inferior Frontal, Middle Temporal, and Motor Cortex , 2017, Cerebral cortex.

[6]  Raffaella Bernardi,et al.  There Is No Logical Negation Here, But There Are Alternatives: Modeling Conversational Negation with Distributional Semantics , 2016, Computational Linguistics.

[7]  Nikolaus Kriegeskorte,et al.  Frontiers in Systems Neuroscience Systems Neuroscience , 2022 .

[8]  Xin Wang,et al.  Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory , 2015, ACL.

[9]  William W. Graves,et al.  Where is the semantic system? A critical review and meta-analysis of 120 functional neuroimaging studies. , 2009, Cerebral cortex.

[10]  Thomas L. Griffiths,et al.  Supplementary Information for Natural Speech Reveals the Semantic Maps That Tile Human Cerebral Cortex , 2022 .

[11]  C. Santamaría,et al.  How negation is understood: Evidence from the visual world paradigm , 2014 .

[12]  Benjamin D. Zinszer,et al.  Representational similarity encoding for fMRI: Pattern-based synthesis to predict brain activity using stimulus-model-similarities , 2016, NeuroImage.

[13]  M. Just,et al.  Changes in activation levels with negation. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[14]  Y. Benjamini,et al.  Controlling the false discovery rate: a practical and powerful approach to multiple testing , 1995 .

[15]  Tom Michael Mitchell,et al.  Predicting Human Brain Activity Associated with the Meanings of Nouns , 2008, Science.

[16]  B. Russell,et al.  “Human Knowledge—Its Scope and Limits” , 1949, Philosophy.

[17]  Massimo Poesio,et al.  Visually Grounded and Textual Semantic Models Differentially Decode Brain Activity Associated with Concrete and Abstract Nouns , 2017, TACL.

[18]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[19]  M. Tettamanti,et al.  Sentential negation of abstract and concrete conceptual categories: a brain decoding multivariate pattern analysis study , 2018, Philosophical Transactions of the Royal Society B: Biological Sciences.

[20]  Brian Murphy,et al.  Simultaneously Uncovering the Patterns of Brain Regions Involved in Different Story Reading Subprocesses , 2014, PloS one.

[21]  Jeroen Geertzen,et al.  The Centre for Speech, Language and the Brain (CSLB) concept property norms , 2013, Behavior research methods.

[22]  Laurence R. Horn,et al.  A brief history of negation , 2010, J. Appl. Log..

[23]  Xinlei Chen,et al.  Visualizing and Understanding Neural Models in NLP , 2015, NAACL.

[24]  Nancy Kanwisher,et al.  Functional specificity for high-level linguistic processing in the human brain , 2011, Proceedings of the National Academy of Sciences.

[25]  M. de Vega,et al.  Sentential Negation Might Share Neurophysiological Mechanisms with Action Inhibition. Evidence from Frontal Theta Rhythm , 2016, The Journal of Neuroscience.

[26]  Tom M. Mitchell,et al.  Selecting Corpus-Semantic Models for Neurolinguistic Decoding , 2012, *SEMEVAL.

[27]  Laurence R. Horn.  A Natural History of Negation , 1989 .

[28]  Massimo Poesio,et al.  Of Words, Eyes and Brains: Correlating Image-Based Distributional Semantic Models with Neural Representations of Concepts , 2013, EMNLP.

[29]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[30]  D. Kemmerer.  Are the motor features of verb meanings represented in the precentral motor cortices? Yes, but within the context of a flexible, multilevel architecture for conceptual knowledge , 2015, Psychonomic Bulletin & Review.

[31]  Christopher Potts,et al.  A Fast Unified Model for Parsing and Sentence Understanding , 2016, ACL.

[32]  Rolf A. Zwaan,et al.  Experiential simulations of negated text information , 2007, Quarterly journal of experimental psychology.

[33]  Andrea Moro,et al.  Negation in the brain: Modulating action representations , 2008, NeuroImage.

[34]  P. Weiss,et al.  To move or not to move: imperatives modulate action-related verb processing in the motor system , 2010, Neuroscience.

[35]  Nancy Kanwisher,et al.  Toward a universal decoder of linguistic meaning from brain activation , 2018, Nature Communications.

[36]  Friedemann Pulvermüller,et al.  Brain mechanisms linking language and action , 2005, Nature Reviews Neuroscience.

[37]  Christopher Potts,et al.  A large annotated corpus for learning natural language inference , 2015, EMNLP.

[38]  Christopher Potts,et al.  Learning Distributed Word Representations for Natural Logic Reasoning , 2014, AAAI Spring Symposia.

[39]  Uri Hasson,et al.  Does understanding negation entail affirmation?: An examination of negated metaphors , 2006 .