Environmental sound priming: Does negation modify N400 cross-modal priming effects?

Human information processing is incredibly fast and flexible. In order to survive, the human brain has to integrate information from various sources and to derive a coherent interpretation, ideally leading to adequate behavior. In experimental setups, such integration phenomena are often investigated in terms of cross-modal association effects. Interestingly, to date, most of these cross-modal association effects using linguistic stimuli have shown that single words can influence the processing of non-linguistic stimuli, and vice versa. In the present study, we were particularly interested in how far linguistic input beyond single words influences the processing of non-linguistic stimuli; in our case, environmental sounds. Participants read sentences either in an affirmative or negated version: for example: “The dog does (not) bark”. Subsequently, participants listened to a sound either matching or mismatching the affirmative version of the sentence (‘woof’ vs. ‘meow’, respectively). In line with previous studies, we found a clear N400-like effect during sound perception following affirmative sentences. Interestingly, this effect was identically present following negated sentences, and the negation operator did not modulate the cross-modal association effect observed between the content words of the sentence and the sound. In summary, these results suggest that negation is not incorporated during information processing in a manner that word–sound association effects would be influenced.

[1]  Mitsuko Aramaki,et al.  Conceptual priming for realistic auditory scenes and for auditory words , 2014, Brain and Cognition.

[2]  Brian Gygi,et al.  How to select stimuli for environmental sound research and where to find them , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[3]  Rolf A. Zwaan,et al.  Processing negated sentences with contradictory predicates: Is a door that is not open mentally closed? , 2006 .

[4]  C. Petten,et al.  Conceptual relationships between spoken words and environmental sounds: Event-related brain potential measures , 1995, Neuropsychologia.

[5]  P. Holcomb,et al.  Imaginal, Semantic, and Surface-Level Processing of Concrete and Abstract Words: An Electrophysiological Investigation , 2000, Journal of Cognitive Neuroscience.

[6]  Brian Gygi,et al.  The incongruency advantage for environmental sounds presented in natural auditory scenes. , 2011, Journal of experimental psychology. Human perception and performance.

[7]  Tad T. Brunyé,et al.  You heard it here first: readers mentally simulate described sounds. , 2010, Acta psychologica.

[8]  Chomsky without language , 1981, Cognition.

[9]  F. Attneave,et al.  The Organization of Behavior: A Neuropsychological Theory , 1949 .

[10]  P. Hagoort,et al.  Integration of Word Meaning and World Knowledge in Language Comprehension , 2004, Science.

[11]  Barbara Kaup,et al.  Is there a difference between stripy journeys and stripy ladybirds? The N400 response to semantic and world-knowledge violations during sentence processing , 2016, Brain and Cognition.

[12]  G. Frege Über Sinn und Bedeutung , 1892 .

[13]  N. Trujillo-Barreto,et al.  Dissociation of the N400 component between linguistic and non-linguistic processing: A source analysis study , 2014 .

[14]  J. H. Howard,et al.  Interpreting the Language of Environmental Sounds , 1987 .

[15]  Tracy Love,et al.  The organization of words and environmental sounds in memory , 2015, Neuropsychologia.

[16]  R. Sperber,et al.  Semantic priming effects on picture and word processing , 1979 .

[17]  J. Townsend,et al.  Auditory semantic networks for words and natural sounds , 2006, Brain Research.

[18]  M. Mesulam,et al.  From sensation to cognition. , 1998, Brain : a journal of neurology.

[19]  M. Kutas,et al.  The Search for Common Sense: An Electrophysiological Study of the Comprehension of Words and Pictures in Reading , 1996, Journal of Cognitive Neuroscience.

[20]  Shravan Vasishth,et al.  The Importance of Reading Naturally: Evidence From Combined Recordings of Eye Movements and Electric Brain Potentials. , 2017, Cognitive science.

[21]  Barbara Kaup,et al.  How Does “Not Left” Become “Right”? Electrophysiological Evidence for a Dynamic Conflict-Bound Negation Processing Account , 2017, Journal of experimental psychology. Human perception and performance.

[22]  Shravan Vasishth,et al.  Brain Responses to World Knowledge Violations: A Comparison of Stimulus- and Fixation-triggered Event-related Potentials and Neural Oscillations , 2015, Journal of Cognitive Neuroscience.

[23]  Barbara Kaup,et al.  The Sounds of Sentences: Differentiating the Influence of Physical Sound, Sound Imagery, and Linguistically Implied Sounds on Physical Sound Processing , 2016, Cognitive, affective & behavioral neuroscience.

[24]  B. Kaup,et al.  Forming associations between language and sensorimotor traces during novel word learning , 2016, Language and Cognition.

[25]  Richard Kronland-Martinet,et al.  The Evocative Power of Sounds: Conceptual Priming between Words and Nonverbal Sounds , 2010, Journal of Cognitive Neuroscience.

[26]  Lonce L. Wyse,et al.  Perceptual and Conceptual Priming of Environmental Sounds , 2011, Journal of Cognitive Neuroscience.

[27]  A. Friederici,et al.  Music, language and meaning: brain signatures of semantic processing , 2004, Nature Neuroscience.

[28]  Guido Orgs,et al.  Conceptual priming for environmental sounds and words: An ERP study , 2006, Brain and Cognition.

[29]  Mante S. Nieuwland,et al.  Quantification, Prediction, and the Online Impact of Sentence Truth-Value: Evidence From Event-Related Potentials , 2015, Journal of experimental psychology. Learning, memory, and cognition.

[30]  M. Kutas,et al.  Brain potentials during reading reflect word expectancy and semantic association , 1984, Nature.

[31]  Guido Orgs,et al.  N400-effects to task-irrelevant environmental sounds: Further evidence for obligatory conceptual processing , 2008, Neuroscience Letters.

[32]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[33]  Mante S. Nieuwland,et al.  If the real world were irrelevant, so to speak: The role of propositional truth-value in counterfactual sentence comprehension , 2012, Cognition.

[34]  Guido Orgs,et al.  Is conceptual priming for environmental sounds obligatory? , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[35]  Salim Roukos,et al.  Brain potentials related to stages of sentence verification. , 1983, Psychophysiology.

[36]  Robert Oostenveld,et al.  Enhanced EEG gamma-band activity reflects multisensory semantic matching in visual-to-auditory object priming , 2008, NeuroImage.

[37]  K. Böcker,et al.  Cortical Measures of Anticipation , 2004 .

[38]  Mante S. Nieuwland,et al.  When the Truth Is Not Too Hard to Handle , 2008, Psychological science.

[39]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..