EEG gamma-band activity during audiovisual speech comprehension in different noise environments

Abstract The presence of cross-modal stochastic resonance in different noise environments has been proved in previous behavioral and event-related potential studies, while it was still unclear whether the gamma-band oscillation study was another evidence of cross-modal stochastic resonance. The multisensory gain of gamma-band activity between the audiovisual (AV) and auditory-only conditions in different noise environments was analyzed. Videos of face motion articulating words concordant with different levels of pink noise were used as stimuli. Signal-to-noise ratios (SNRs) of 0, −4, −8, −12 and −16 dB were selected to measure the speech recognition accuracy and EEG activity for 20 healthy subjects. The power and phase of EEG gamma-band oscillations increased in a time window of 50–90 ms. The multisensory gains of evoked and total activity, as well as phase-locking factor, were greatest at the −12 dB SNR, which were consistent with the behavioral result. The multisensory gain of gamma-band activity showed an inverted U-shaped curve as a function of SNR. This finding confirmed the presence of cross-modal stochastic resonance. In addition, there was a significant correlation between evoked activity and phase-locking factor of gamma-band at five different SNRs. Gamma-band oscillation was believed to play a role in the rapid processing and information linkage strengthening of AV modalities in the early stage of cognitive processes.

[1]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[2]  A. Macleod,et al.  Quantifying the contribution of vision to speech perception in noise. , 1987, British journal of audiology.

[3]  J. Pernier,et al.  Stimulus Specificity of Phase-Locked and Non-Phase-Locked 40 Hz Visual Responses in Human , 1996, The Journal of Neuroscience.

[4]  E. Basar,et al.  Gamma-band responses in the brain: a short review of psychophysiological correlates and functional significance. , 1996, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[5]  Catherine Tallon-Baudry,et al.  Induced γ-Band Activity during the Delay of a Visual Short-Term Memory Task in Humans , 1998, The Journal of Neuroscience.

[6]  J. Pernier,et al.  Induced gamma-band activity during the delay of a visual short-term memory task in humans. , 1998, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[7]  Peter Grigg,et al.  Effects of Colored Noise on Stochastic Resonance in Sensory Neurons , 1999 .

[8]  W. Singer,et al.  Dynamic predictions: Oscillations and synchrony in top–down processing , 2001, Nature Reviews Neuroscience.

[9]  D E Callan,et al.  Multimodal contribution to speech perception revealed by independent component analysis: a single-sweep EEG case study. , 2001, Brain research. Cognitive brain research.

[10]  R. McCarley,et al.  Abnormal Neural Synchrony in Schizophrenia , 2003, The Journal of Neuroscience.

[11]  Jeffery A. Jones,et al.  Neural processes underlying perceptual enhancement by visual speech gestures , 2003, Neuroreport.

[12]  W. Singer,et al.  Phase Sensitivity of Synaptic Modifications in Oscillating Cells of Rat Visual Cortex , 2004, The Journal of Neuroscience.

[13]  A. Engel,et al.  Cognitive functions of gamma-band activity: memory match and utilization , 2004, Trends in Cognitive Sciences.

[14]  R. McCarley,et al.  Neural synchrony indexes disordered perception and cognition in schizophrenia. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[15]  V I Nekorkin,et al.  Self-referential phase reset based on inferior olive oscillator dynamics , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[16]  G. Buzsáki,et al.  Neuronal Oscillations in Cortical Networks , 2004, Science.

[17]  Werner Lutzenberger,et al.  Cortical Oscillatory Activity and the Dynamics of Auditory Memory Processing , 2005, Reviews in the neurosciences.

[18]  Henning Scheich,et al.  Learning-induced plasticity in animal and human auditory cortex , 2005, Current Opinion in Neurobiology.

[19]  P. Hagoort,et al.  Oscillatory neuronal dynamics during language comprehension. , 2006, Progress in brain research.

[20]  R. Oostenveld,et al.  Tactile Spatial Attention Enhances Gamma-Band Activity in Somatosensory Cortex and Reduces Low-Frequency Activity in Parieto-Occipital Areas , 2006, The Journal of Neuroscience.

[21]  F Xavier Castellanos,et al.  Brain development and ADHD. , 2006, Clinical psychology review.

[22]  G. Dawson,et al.  Resting State Cortical Connectivity Reflected in EEG Coherence in Individuals With Autism , 2007, Biological Psychiatry.

[23]  Christoph S. Herrmann,et al.  EEG oscillations in the gamma and alpha range respond differently to spatial frequency , 2007, Vision Research.

[24]  W. Singer,et al.  The gamma cycle , 2007, Trends in Neurosciences.

[25]  Daniel Senkowski,et al.  Good times for multisensory integration: Effects of the precision of temporal synchrony as revealed by gamma-band oscillations , 2007, Neuropsychologia.

[26]  J. Kaiser,et al.  Human gamma-frequency oscillations associated with attention and memory , 2007, Trends in Neurosciences.

[27]  Christoph S. Herrmann,et al.  Evoked γ oscillations in human scalp EEG are test–retest reliable , 2007, Clinical Neurophysiology.

[28]  Shlomit Yuval-Greenberg,et al.  What You See Is Not (Always) What You Hear: Induced Gamma Band Responses Reflect Cross-Modal Interactions in Familiar Object Recognition , 2007, The Journal of Neuroscience.

[29]  Ingo Fründ,et al.  Evoked gamma oscillations in human scalp EEG are test-retest reliable. , 2007, Clinical neurophysiology : official journal of the International Federation of Clinical Neurophysiology.

[30]  John J. Foxe,et al.  Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. , 2006, Cerebral cortex.

[31]  Claudio R. Mirasso,et al.  Effects of auditory noise on the psychophysical detection of visual signals: Cross-modal stochastic resonance , 2007, Neuroscience Letters.

[32]  John J. Foxe,et al.  Impaired multisensory processing in schizophrenia: Deficits in the visual enhancement of speech comprehension under noisy environmental conditions , 2007, Schizophrenia Research.

[33]  L. Merabet,et al.  Visual Phosphene Perception Modulated by Subthreshold Crossmodal Sensory Stimulation , 2007, The Journal of Neuroscience.

[34]  S. Aou,et al.  Dynamical properties of the two-process model for sleep-wake cycles in infantile autism , 2008, Cognitive Neurodynamics.

[35]  N. Logothetis,et al.  Visual modulation of neurons in auditory cortex. , 2008, Cerebral cortex.

[36]  John J. Foxe,et al.  Crossmodal binding through neural coherence: implications for multisensory processing , 2008, Trends in Neurosciences.

[37]  Werner Lutzenberger,et al.  Distinct Gamma-Band Components Reflect the Short-Term Memory Maintenance of Different Sound Lateralization Angles , 2008, Cerebral cortex.

[38]  Robert Oostenveld,et al.  Enhanced EEG gamma-band activity reflects multisensory semantic matching in visual-to-auditory object priming , 2008, NeuroImage.

[39]  J. E. Lugo,et al.  Multisensory Integration , 2008, Psychological science.

[40]  Rafael Doti,et al.  Ubiquitous Crossmodal Stochastic Resonance in Humans: Auditory Noise Facilitates Tactile, Visual and Proprioceptive Sensations , 2008, PloS one.

[41]  Christoph S Herrmann,et al.  Modes of memory: early electrophysiological markers of repetition suppression and recognition enhancement predict behavioral performance. , 2007, Psychophysiology.

[42]  C. Tallon-Baudry,et al.  Neural Dissociation between Visual Awareness and Spatial Attention , 2008, The Journal of Neuroscience.

[43]  Donald A. Glaser,et al.  Spontaneous Local Gamma Oscillation Selectively Enhances Neural Network Responsiveness , 2009, PLoS Comput. Biol..

[44]  Lee M. Miller,et al.  A Multisensory Cortical Network for Understanding Speech in Noise , 2009, Journal of Cognitive Neuroscience.

[45]  Ingo Fründ,et al.  Human gamma-band activity: A review on cognitive and behavioral correlates and network models , 2010, Neuroscience & Biobehavioral Reviews.

[46]  Herrmann Christoph,et al.  Gamma amplitudes are coupled to theta phase in human EEG during visual perception , 2010 .

[47]  Axel H. Winneke,et al.  Does audiovisual speech offer a fountain of youth for old ears? An event-related brain potential study of age differences in audiovisual speech perception. , 2011, Psychology and aging.

[48]  René J. Huster,et al.  Auditory Event-Related Response in Visual Cortex Modulates Subsequent Visual Responses in Humans , 2011, The Journal of Neuroscience.

[49]  Włodzisław Duch,et al.  Visualization for understanding of neurodynamical systems , 2011, Cognitive Neurodynamics.

[50]  Daniel Mirman,et al.  Eye movement dynamics and cognitive self-organization in typical and atypical development , 2012, Cognitive Neurodynamics.

[51]  Aina Puce,et al.  Inverse Effectiveness and Multisensory Interactions in Visual Event-Related Potentials with Audiovisual Speech , 2012, Brain Topography.

[52]  B. Liu,et al.  Correlation between audio–visual enhancement of speech in different noise environments and SNR: A combined behavioral and electrophysiological study , 2013, Neuroscience.

[53]  Wei Wu,et al.  Extraction of mismatch negativity using a resampling-based spatial filtering method , 2013, Journal of neural engineering.

[54]  Frank Huethe,et al.  Broad-band Gaussian noise is most effective in improving motor performance and is most pleasant , 2014, Front. Hum. Neurosci..