The organization of words and environmental sounds in memory

In the present study we used event-related potentials to compare the organization of linguistic and meaningful nonlinguistic sounds in memory. We examined N400 amplitudes as adults viewed pictures presented with words or environmental sounds that matched the picture (Match), that shared semantic features with the expected match (Near Violation), and that shared relatively few semantic features with the expected match (Far Violation). Words demonstrated incremental N400 amplitudes based on featural similarity from 300-700ms, such that both Near and Far Violations exhibited significant N400 effects, however Far Violations exhibited greater N400 effects than Near Violations. For environmental sounds, Far Violations but not Near Violations elicited significant N400 effects, in both early (300-400ms) and late (500-700ms) time windows, though a graded pattern similar to that of words was seen in the mid-latency time window (400-500ms). These results indicate that the organization of words and environmental sounds in memory is differentially influenced by featural similarity, with a consistently fine-grained graded structure for words but not sounds.

[1]  Brian Gygi,et al.  How to select stimuli for environmental sound research and where to find them , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[2]  Mitsuko Aramaki,et al.  Conceptual priming for realistic auditory scenes and for auditory words , 2014, Brain and Cognition.

[3]  C. Petten,et al.  Conceptual relationships between spoken words and environmental sounds: Event-related brain potential measures , 1995, Neuropsychologia.

[4]  M. Kutas Event related brain potential studies of language , 1988 .

[5]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[6]  F. Dick,et al.  An on-line task for contrasting auditory processing in the verbal and nonverbal domains and norms for younger and older adults , 2005, Behavior research methods.

[7]  Magnus Lindgren,et al.  Semantic organization of basic-level words in 20-month-olds: An ERP study , 2006, Journal of Neurolinguistics.

[8]  Dylan M. Jones,et al.  Priming the Identification of Environmental Sounds , 1995, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[9]  Guillaume Thierry,et al.  Hemispheric Dissociation in Access to the Human Semantic System , 2003, Neuron.

[10]  J. Polich Neuropsychology of P300 , 2011 .

[11]  J. H. Howard,et al.  Interpreting the Language of Environmental Sounds , 1987 .

[12]  Mark S. Seidenberg,et al.  Semantic feature production norms for a large set of living and nonliving things , 2005, Behavior research methods.

[13]  Doreen Kimura,et al.  From ear to brain , 2011, Brain and Cognition.

[14]  S. Koelsch Neural substrates of processing syntax and semantics in music , 2005, Current Opinion in Neurobiology.

[15]  René van Egmond,et al.  The effect of visual context on the identification of ambiguous environmental sounds , 2009 .

[16]  Guido Orgs,et al.  N400-effects to task-irrelevant environmental sounds: Further evidence for obligatory conceptual processing , 2008, Neuroscience Letters.

[17]  M. Kutas,et al.  Event-related brain potentials to grammatical errors and semantic anomalies , 1983, Memory & cognition.

[18]  John E Richards,et al.  Attention affects the recognition of briefly presented visual stimuli in infants: an ERP study. , 2003, Developmental science.

[19]  J. Polich,et al.  Comparison of P300 from passive and active tasks for auditory and visual stimuli. , 1999, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[20]  G. Kuperberg,et al.  Multiple Influences of Semantic Memory on Sentence Processing: Distinct Effects of Semantic Relatedness on Violations of Real-World Event/State Knowledge and Animacy Selection Restrictions. , 2012, Journal of memory and language.

[21]  Stefan Debener,et al.  Multisensory identification of natural objects in a two-way crossmodal priming paradigm. , 2008, Experimental psychology.

[22]  Michael J Cortese,et al.  Handbook of Psycholinguistics , 2011 .

[23]  Karl J. Friston,et al.  The effect of prior visual information on recognition of speech and sounds. , 2008, Cerebral cortex.

[24]  Ole Jensen,et al.  Oxford handbook of event-related potential components. , 2011 .

[25]  Guido Orgs,et al.  Is conceptual priming for environmental sounds obligatory? , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[26]  Richard Kronland-Martinet,et al.  Sound Categorization and Conceptual Priming for Nonlinguistic and Linguistic Sounds , 2010, Journal of Cognitive Neuroscience.

[27]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[28]  J. Hocking,et al.  NESSTI: Norms for Environmental Sound Stimuli , 2013, PloS one.

[29]  Kara D. Federmeier,et al.  A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing , 1999 .

[30]  Keiji Tanaka,et al.  Representation of Visual Features of Objects in the Inferotemporal Cortex , 1996, Neural Networks.

[31]  Charles Spence,et al.  Crossmodal semantic priming by naturalistic sounds and spoken words enhances visual sensitivity. , 2011, Journal of experimental psychology. Human perception and performance.

[32]  Marta Kutas,et al.  The impact of semantic memory organization and sentence context information on spoken language processing by younger and older adults: an ERP study. , 2002, Psychophysiology.

[33]  E. DeYoe,et al.  Distinct Cortical Pathways for Processing Tool versus Animal Sounds , 2005, The Journal of Neuroscience.

[34]  Ulrich H. Frauenfelder,et al.  The process of spoken word recognition: An introduction , 1987, Cognition.

[35]  J. Townsend,et al.  Auditory semantic networks for words and natural sounds , 2006, Brain Research.

[36]  C. Connine,et al.  Semantic richness: The role of semantic features in processing spoken words , 2014 .

[37]  G. R. Eichhorn,et al.  Neural correlates of naming animals from their characteristic sounds , 2003, Neuropsychologia.

[38]  J. Hampton,et al.  Semantic Memory Redux: An Experimental Test of Hierarchical Category Representation. , 2012, Journal of memory and language.

[39]  C. Price,et al.  Speech-specific auditory processing: where is it? , 2005, Trends in Cognitive Sciences.

[40]  P. Kay Taxonomy and Semantic Contrast , 1971 .

[41]  R. Čeponienė,et al.  Verbal and nonverbal semantic processing in children with developmental language impairment , 2010, Neuropsychologia.

[42]  G Hickok,et al.  Role of anterior temporal cortex in auditory sentence comprehension: an fMRI study , 2001, Neuroreport.

[43]  Jérôme Daltrozzo,et al.  Conceptual Processing in Music as Revealed by N400 Effects on Words and Musical Targets , 2009, Journal of Cognitive Neuroscience.

[44]  E. T. Possing,et al.  Human temporal lobe activation by speech and nonspeech sounds. , 2000, Cerebral cortex.

[45]  Riitta Salmelin,et al.  Parametric merging of MEG and fMRI reveals spatiotemporal differences in cortical processing of spoken words and environmental sounds in background noise. , 2012, Cerebral cortex.

[46]  H. Neville,et al.  Language and , 2019, Adventure Diffusion.

[47]  A. Friederici,et al.  Music, language and meaning: brain signatures of semantic processing , 2004, Nature Neuroscience.

[48]  Vladimir López,et al.  ERPs and contextual semantic discrimination: Degrees of congruence in wakefulness and sleep , 2006, Brain and Language.

[49]  A. Saygin,et al.  Distributed processing and cortical specialization for speech and environmental sounds in human temporal cortex , 2011, Brain and Language.

[50]  Guido Orgs,et al.  Conceptual priming for environmental sounds and words: An ERP study , 2006, Brain and Cognition.

[51]  J. Ballas Common factors in the identification of an assortment of brief everyday sounds , 1993 .

[52]  J. Townsend,et al.  A developmental ERP study of verbal and non-verbal semantic processing , 2008, Brain Research.

[53]  W. Maki,et al.  Associative priming by targets and distractors during rapid serial visual presentation: does word meaning survive the attentional blink? , 1997, Journal of experimental psychology. Human perception and performance.

[54]  W. Levelt,et al.  The spatial and temporal signatures of word production components , 2004, Cognition.

[55]  Brian Gygi,et al.  Factors in the Identification of Environmental Sounds , 2001 .

[56]  John Polich,et al.  Semantic categorization and event-related potentials , 1985, Brain and Language.

[57]  E. Vogel,et al.  Electrophysiological Evidence for a Postperceptual Locus of Suppression during the Attentional Blink Time-based Attention and the Attentional Blink , 1998 .

[58]  Richard Kronland-Martinet,et al.  The Evocative Power of Sounds: Conceptual Priming between Words and Nonverbal Sounds , 2010, Journal of Cognitive Neuroscience.

[59]  Lonce L. Wyse,et al.  Perceptual and Conceptual Priming of Environmental Sounds , 2011, Journal of Cognitive Neuroscience.

[60]  Katrin Amunts,et al.  Architecture and organizational principles of Broca's region , 2012, Trends in Cognitive Sciences.

[61]  Ava J. Senkfor,et al.  Electrophysiological dissociation between verbal and nonverbal semantic processing in learning disabled adults , 2000, Neuropsychologia.

[62]  D. Kimura,et al.  Cerebral processing of nonverbal sounds in boys and girls. , 1970, Neuropsychologia.

[63]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[64]  Aaron J. Newman,et al.  The Influence of Language Proficiency on Lexical Semantic Processing in Native and Late Learners of English , 2012, Journal of Cognitive Neuroscience.

[65]  David Friedman,et al.  Effect of Sound Familiarity on the Event-Related Potentials Elicited by Novel Environmental Sounds , 1998, Brain and Cognition.

[66]  D. Hubel,et al.  Laminar and columnar distribution of geniculo‐cortical fibers in the macaque monkey , 1972, The Journal of comparative neurology.

[67]  Kara D. Federmeier,et al.  Picture the difference: electrophysiological investigations of picture processing in the two cerebral hemispheres , 2002, Neuropsychologia.

[68]  Stefan Koelsch,et al.  Comparing the Processing of Music and Language Meaning Using EEG and fMRI Provides Evidence for Similar and Distinct Neural Representations , 2008, PloS one.

[69]  Raymond D. Kent,et al.  Articulatory timing in selected consonant sequences , 1975, Brain and Language.

[70]  E. Vogel,et al.  Word meanings can be accessed but not reported during the attentional blink , 1996, Nature.