Mismatch Negativity with Visual-only and Audiovisual Speech

The functional organization of cortical speech processing is thought to be hierarchical, increasing in complexity and proceeding from primary sensory areas centrifugally. The current study used the mismatch negativity (MMN) obtained with electrophysiology (EEG) to investigate the early latency period of visual speech processing under both visual-only (VO) and audiovisual (AV) conditions. Current density reconstruction (CDR) methods were used to model the cortical MMN generator locations. MMNs were obtained with VO and AV speech stimuli at early latencies (approximately 82–87 ms peak in time waveforms relative to the acoustic onset) and in regions of the right lateral temporal and parietal cortices. Latencies were consistent with bottom-up processing of the visible stimuli. We suggest that a visual pathway extracts phonetic cues from visible speech, and that previously reported effects of AV speech in classical early auditory areas, given later reported latencies, could be attributable to modulatory feedback from visual phonetic processing.

[1]  H. Jasper,et al.  The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. , 1999, Electroencephalography and clinical neurophysiology. Supplement.

[2]  G. A. Calvert,et al.  Auditory-visual processing represented in the human superior temporal gyrus , 2007, Neuroscience.

[3]  Lynne E. Bernstein,et al.  Spatiotemporal dynamics of audiovisual speech processing , 2008, NeuroImage.

[4]  Riadh Lebib,et al.  Modulations of ‘late’ event-related brain potentials in humans by dynamic audiovisual speech stimuli , 2004, Neuroscience Letters.

[5]  Mikko Sams,et al.  Visual Processing Affects the Neural Basis of Auditory Discrimination , 2008, Journal of Cognitive Neuroscience.

[6]  P. Deltenre,et al.  Mismatch negativity evoked by the McGurk–MacDonald effect: a phonetic representation within short-term memory , 2002, Clinical Neurophysiology.

[7]  M. Fuchs,et al.  Linear and nonlinear current density reconstructions. , 1999, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[8]  R. Blake,et al.  Brain Areas Involved in Perception of Biological Motion , 2000, Journal of Cognitive Neuroscience.

[9]  T Allison,et al.  ERPS EVOKED BY VIEWING FACIAL MOVEMENTS , 2000, Cognitive neuropsychology.

[10]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[11]  I. Winkler,et al.  Memory-based or afferent processes in mismatch negativity (MMN): a review of the evidence. , 2005, Psychophysiology.

[12]  T. Allison,et al.  Temporal Cortex Activation in Humans Viewing Eye and Mouth Movements , 1998, The Journal of Neuroscience.

[13]  F. Perrin,et al.  Brain generators implicated in the processing of auditory stimulus deviance: a topographic event-related potential study. , 1990, Psychophysiology.

[14]  R. Näätänen,et al.  The mismatch negativity (MMN) in basic research of central auditory processing: A review , 2007, Clinical Neurophysiology.

[15]  R. Hari,et al.  Seeing speech: visual information from lip movements modifies activity in the human auditory cortex , 1991, Neuroscience Letters.

[16]  P. Deltenre,et al.  Generalization of the generation of an MMN by illusory McGurk percepts: voiceless consonants , 2004, Clinical Neurophysiology.

[17]  Piia Astikainen,et al.  Visual mismatch negativity for changes in orientation – a sensory memory‐dependent response , 2008, The European journal of neuroscience.

[18]  John J. Foxe,et al.  Seeing voices: High-density electrical mapping and source-analysis of the multisensory mismatch negativity evoked during the McGurk illusion , 2007, Neuropsychologia.

[19]  J. Pernier,et al.  A systematic evaluation of the spherical model accuracy in EEG dipole localization. , 1997, Electroencephalography and clinical neurophysiology.

[20]  G. Romani,et al.  Auditory evoked magnetic fields and electric potentials , 1990 .

[21]  Lynne E. Bernstein,et al.  Neurocognitive basis for audiovisual speech perception: evidence from event-related potentials , 2002, INTERSPEECH.

[22]  E. Bullmore,et al.  Response amplification in sensory-specific cortices during crossmodal binding. , 1999, Neuroreport.

[23]  S. Scott,et al.  The neuroanatomical and functional organization of speech perception , 2003, Trends in Neurosciences.

[24]  M. Fuchs,et al.  Spatio-Temporal Current Density Reconstruction (stCDR) from EEG/MEG-Data , 2004, Brain Topography.

[25]  R. Näätänen,et al.  Early selective-attention effect on evoked potential reinterpreted. , 1978, Acta psychologica.

[26]  R D Pascual-Marqui,et al.  Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. , 2002, Methods and findings in experimental and clinical pharmacology.

[27]  D. Lehmann,et al.  Reference-free identification of components of checkerboard-evoked multichannel potential fields. , 1980, Electroencephalography and clinical neurophysiology.

[28]  Aina Puce,et al.  Electrophysiology and brain imaging of biological motion. , 2003, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[29]  Cheryl M. Capek,et al.  Cortical circuits for silent speechreading in deaf and hearing people , 2008, Neuropsychologia.

[30]  Mikko Sams,et al.  Processing of changes in visual speech in the human auditory cortex. , 2002, Brain research. Cognitive brain research.

[31]  John J. Foxe,et al.  The neural circuitry of pre-attentive auditory change-detection: an fMRI study of pitch and duration mismatch negativity generators. , 2005, Cerebral cortex.

[32]  L. Bernstein,et al.  Quantified acoustic–optical speech signal incongruity identifies cortical sites of audiovisual speech processing , 2008, Brain Research.

[33]  István Czigler,et al.  Backward masking and visual mismatch negativity: electrophysiological evidence for memory-based detection of deviant stimuli. , 2007, Psychophysiology.

[34]  E. Vatikiotis-Bateson,et al.  Perceiving Biological Motion: Dissociating Visible Speech from Walking , 2003, Journal of Cognitive Neuroscience.

[35]  I. Winkler,et al.  MMN or no MMN: no magnitude of deviance effect on the MMN amplitude. , 2007, Psychophysiology.

[36]  E. Amenedo,et al.  MMN in the visual modality: a review , 2003, Biological Psychology.

[37]  R. Campbell,et al.  Reading Speech from Still and Moving Faces: The Neural Substrates of Visible Speech , 2003, Journal of Cognitive Neuroscience.

[38]  J. Eggermont,et al.  Integrated mismatch negativity (MMNi): a noise-free representation of evoked responses allowing single-point distribution-free statistical tests. , 1997, Electroencephalography and clinical neurophysiology.