Enhanced EEG gamma-band activity reflects multisensory semantic matching in visual-to-auditory object priming

An important step in perceptual processing is the integration of information from different sensory modalities into a coherent percept. It has been suggested that such crossmodal binding might be achieved by transient synchronization of neurons from different modalities in the gamma-frequency range (>30 Hz). Here we employed a crossmodal priming paradigm, modulating the semantic congruency between visual-auditory natural object stimulus pairs, during the recording of the high density electroencephalogram (EEG). Subjects performed a semantic categorization task. Analysis of the behavioral data showed a crossmodal priming effect (facilitated auditory object recognition) in response to semantically congruent stimuli. Differences in event-related potentials (ERP) were found between 250 and 350 ms, which were localized to left middle temporal gyrus (BA 21) using a distributed linear source model. Early gamma-band activity (40-50 Hz) was increased between 120 ms and 180 ms following auditory stimulus onset for semantically congruent stimulus pairs. Source reconstruction for this gamma-band response revealed a maximal increase in left middle temporal gyrus (BA 21), an area known to be related to the processing of both complex auditory stimuli and multisensory processing. The data support the hypothesis that oscillatory activity in the gamma-band reflects crossmodal semantic-matching processes in multisensory convergence sites.

[1]  Lucas Spierer,et al.  Plasticity in representations of environmental sounds revealed by electrical neuroimaging , 2008, NeuroImage.

[2]  A. Giraud,et al.  Implicit Multisensory Associations Influence Voice Recognition , 2006, PLoS biology.

[3]  Robert Oostenveld,et al.  Localizing human visual gamma-band activity in frequency, time and space , 2006, NeuroImage.

[4]  John J. Foxe,et al.  Multisensory processing of naturalistic objects in motion: A high-density electrical mapping and source estimation study , 2007, NeuroImage.

[5]  C. Petten,et al.  Conceptual relationships between spoken words and environmental sounds: Event-related brain potential measures , 1995, Neuropsychologia.

[6]  C. Petten,et al.  Neural localization of semantic context effects in electromagnetic and hemodynamic studies , 2006, Brain and Language.

[7]  Karl J. Friston,et al.  The effect of prior visual information on recognition of speech and sounds. , 2008, Cerebral cortex.

[8]  W Singer,et al.  Visual feature integration and the temporal correlation hypothesis. , 1995, Annual review of neuroscience.

[9]  Christoph M. Michel,et al.  Rapid discrimination of visual and multisensory memories revealed by electrical neuroimaging , 2004, NeuroImage.

[10]  W. Drongelen,et al.  Localization of brain electrical activity via linearly constrained minimum variance spatial filtering , 1997, IEEE Transactions on Biomedical Engineering.

[11]  Jean-Philippe Thiran,et al.  What and Where in human audition: selective deficits following focal hemispheric lesions , 2002, Experimental Brain Research.

[12]  O. Bertrand,et al.  Oscillatory gamma activity in humans and its role in object representation , 1999, Trends in Cognitive Sciences.

[13]  D. Guthrie,et al.  Significance testing of difference potentials. , 1991, Psychophysiology.

[14]  J. Thiran,et al.  Distinct Pathways Involved in Sound Recognition and Localization: A Human fMRI Study , 2000, NeuroImage.

[15]  Micah M. Murray,et al.  Rapid Brain Discrimination of Sounds of Objects , 2006, The Journal of Neuroscience.

[16]  C. Michel,et al.  Noninvasive Localization of Electromagnetic Epileptic Activity. II. Demonstration of Sublobar Accuracy in Patients with Simultaneous Surface and Depth Recordings , 2004, Brain Topography.

[17]  K. Grill-Spector,et al.  Repetition and the brain: neural models of stimulus-specific effects , 2006, Trends in Cognitive Sciences.

[18]  D. Kourtis,et al.  Neurophysiology of Implicit Timing in Serial Choice Reaction-Time Performance , 2006, The Journal of Neuroscience.

[19]  M. Tervaniemi,et al.  Binding symbols and sounds: evidence from event-related oscillatory gamma-band activity. , 2007, Cerebral cortex.

[20]  D. Schacter,et al.  Cortical activity reductions during repetition priming can result from rapid response learning , 2004, Nature.

[21]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[22]  S. Petersen,et al.  Memory's echo: vivid remembering reactivates sensory-specific cortex. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[23]  Paul J. Laurienti,et al.  Semantic congruence is a critical factor in multisensory behavioral performance , 2004, Experimental Brain Research.

[24]  Jonathan Grainger,et al.  An electrophysiological study of cross-modal repetition priming. , 2005, Psychophysiology.

[25]  A. Engel,et al.  High-frequency activity in human visual cortex is modulated by visual motion strength. , 2007, Cerebral cortex.

[26]  S. Debener,et al.  Late auditory evoked potentials asymmetry revisited , 2007, Clinical Neurophysiology.

[27]  Jeffrey R Binder,et al.  Human brain regions involved in recognizing environmental sounds. , 2004, Cerebral cortex.

[28]  Guido Orgs,et al.  Conceptual priming for environmental sounds and words: An ERP study , 2006, Brain and Cognition.

[29]  Daniel Lenz,et al.  What's that sound? Matches with auditory long-term memory induce gamma activity in human EEG. , 2007, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[30]  R. Oostenveld,et al.  Tactile Spatial Attention Enhances Gamma-Band Activity in Somatosensory Cortex and Reduces Low-Frequency Activity in Parieto-Occipital Areas , 2006, The Journal of Neuroscience.

[31]  J. Bosch,et al.  N400 during lexical decision tasks: a current source localization study , 2003, Clinical Neurophysiology.

[32]  L. Benevento,et al.  Auditory-visual interaction in single cells in the cortex of the superior temporal sulcus and the orbital frontal cortex of the macaque monkey , 1977, Experimental Neurology.

[33]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[34]  Kara D. Federmeier,et al.  Electrophysiology reveals semantic memory use in language comprehension , 2000, Trends in Cognitive Sciences.

[35]  W. Singer,et al.  Oscillatory responses in cat visual cortex exhibit inter-columnar synchronization which reflects global stimulus properties , 1989, Nature.

[36]  Daniel Senkowski,et al.  Multisensory processing and oscillatory gamma responses: effects of spatial selective attention , 2005, Experimental Brain Research.

[37]  R. Meuli,et al.  Auditory agnosia and auditory spatial deficits following left hemispheric lesions: evidence for distinct processing pathways , 2000, Neuropsychologia.

[38]  R. Zatorre,et al.  Sensitivity to Auditory Object Features in Human Temporal Neocortex , 2004, The Journal of Neuroscience.

[39]  Ingo Fründ,et al.  Stimulus intensity affects early sensory processing: sound intensity modulates auditory evoked gamma-band activity in human EEG. , 2007, International Journal of Psychophysiology.

[40]  E. DeYoe,et al.  Distinct Cortical Pathways for Processing Tool versus Animal Sounds , 2005, The Journal of Neuroscience.

[41]  Christoph M. Michel,et al.  Electrical neuroimaging based on biophysical constraints , 2004, NeuroImage.

[42]  Matthias M. Müller,et al.  Effects of picture repetition on induced gamma band responses, evoked potentials, and phase synchrony in the human EEG. , 2002, Brain research. Cognitive brain research.

[43]  D. Lehmann,et al.  Reference-free identification of components of checkerboard-evoked multichannel potential fields. , 1980, Electroencephalography and clinical neurophysiology.

[44]  John J. Foxe,et al.  Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. , 2004, Cerebral cortex.

[45]  Ch. von der Malsburg,et al.  A neural cocktail-party processor , 1986, Biological Cybernetics.

[46]  J Gross,et al.  REPRINTS , 1962, The Lancet.

[47]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[49]  Matthias M. Müller,et al.  Oscillatory brain activity dissociates between associative stimulus content in a repetition priming task in the human EEG. , 2004, Cerebral cortex.

[50]  P König,et al.  Direct physiological evidence for scene segmentation by temporal coding. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[51]  W. Singer,et al.  Dynamic predictions: Oscillations and synchrony in top–down processing , 2001, Nature Reviews Neuroscience.

[52]  R. Desimone,et al.  Visual properties of neurons in a polysensory area in superior temporal sulcus of the macaque. , 1981, Journal of neurophysiology.

[53]  John J. Foxe,et al.  Crossmodal binding through neural coherence: implications for multisensory processing , 2008, Trends in Neurosciences.

[54]  Michael S Beauchamp,et al.  See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex , 2005, Current Opinion in Neurobiology.

[55]  A. Engel,et al.  Cognitive functions of gamma-band activity: memory match and utilization , 2004, Trends in Cognitive Sciences.

[56]  Shlomit Yuval-Greenberg,et al.  What You See Is Not (Always) What You Hear: Induced Gamma Band Responses Reflect Cross-Modal Interactions in Familiar Object Recognition , 2007, The Journal of Neuroscience.

[57]  Alex Martin,et al.  Properties and mechanisms of perceptual priming , 1998, Current Opinion in Neurobiology.

[58]  Stefan Debener,et al.  Multisensory identification of natural objects in a two-way crossmodal priming paradigm. , 2008, Experimental psychology.

[59]  C. Michel,et al.  Noninvasive Localization of Electromagnetic Epileptic Activity. I. Method Descriptions and Simulations , 2004, Brain Topography.

[60]  M. Kutas,et al.  Reading senseless sentences: brain potentials reflect semantic incongruity. , 1980, Science.

[61]  A. Dale,et al.  Functional-Anatomic Correlates of Object Priming in Humans Revealed by Rapid Presentation Event-Related fMRI , 1998, Neuron.

[62]  John J. Foxe,et al.  Object‐based attention is multisensory: co‐activation of an object's representations in ignored sensory modalities , 2007, The European journal of neuroscience.

[63]  P König,et al.  Synchronization of oscillatory neuronal responses between striate and extrastriate visual cortical areas of the cat. , 1991, Proceedings of the National Academy of Sciences of the United States of America.

[64]  A. Engel,et al.  What is novel in the novelty oddball paradigm? Functional significance of the novelty P3 event-related potential as revealed by independent component analysis. , 2005, Brain research. Cognitive brain research.

[65]  A. Amedi,et al.  Functional imaging of human crossmodal identification and object recognition , 2005, Experimental Brain Research.

[66]  J. Martinerie,et al.  The brainweb: Phase synchronization and large-scale integration , 2001, Nature Reviews Neuroscience.

[67]  Werner Lutzenberger,et al.  Hearing lips: gamma-band activity during audiovisual speech perception. , 2005, Cerebral cortex.

[68]  Christoph S. Herrmann,et al.  Time-frequency analysis of target detection reveals an early interface between bottom-up and top-down processes in the gamma-band , 2006, NeuroImage.

[69]  Stefan Debener,et al.  Size matters: effects of stimulus size, duration and eccentricity on the visual gamma-band response , 2004, Clinical Neurophysiology.

[70]  Daniel Senkowski,et al.  Good times for multisensory integration: Effects of the precision of temporal synchrony as revealed by gamma-band oscillations , 2007, Neuropsychologia.

[71]  Daniel Gembris,et al.  Top-down attentional processing enhances auditory evoked gamma band activity , 2003, Neuroreport.

[72]  Thomas Grunwald,et al.  Neural Bases of Cognitive ERPs: More than Phase Reset , 2004, Journal of Cognitive Neuroscience.