Tuning and disrupting the brain—modulating the McGurk illusion with electrical stimulation

In the so-called McGurk illusion, when the synchronized presentation of the visual stimulus /ga/ is paired with the auditory stimulus /ba/, people in general hear it as /da/. Multisensory integration processing underlying this illusion seems to occur within the Superior Temporal Sulcus (STS). Herein, we present evidence demonstrating that bilateral cathodal transcranial direct current stimulation (tDCS) of this area can decrease the McGurk illusion-type responses. Additionally, we show that the manipulation of this audio-visual integrated output occurs irrespective of the number of eye-fixations on the mouth of the speaker. Bilateral anodal tDCS of the Parietal Cortex also modulates the illusion, but in the opposite manner, inducing more illusion-type responses. This is the first demonstration of using non-invasive brain stimulation to modulate multisensory speech perception in an illusory context (i.e., both increasing and decreasing illusion-type responses to a verbal audio-visual integration task). These findings provide clear evidence that both the superior temporal and parietal areas contribute to multisensory integration processing related to speech perception. Specifically, STS seems fundamental for the temporal synchronization and integration of auditory and visual inputs. For its part, posterior parietal cortex (PPC) may adjust the arrival of incoming audio and visual information to STS thereby enhancing their interaction in this latter area.

[1]  Hans-Jochen Heinze,et al.  Coding of multisensory temporal patterns in human superior temporal sulcus , 2012, Front. Integr. Neurosci..

[2]  Felipe Fregni,et al.  Polarity-Dependent Transcranial Direct Current Stimulation Effects on Central Auditory Processing , 2011, PloS one.

[3]  Elena Plante,et al.  Auditory-visual speech integration by adults with and without language-learning disabilities. , 2006, Journal of communication disorders.

[4]  Michael J Brammer,et al.  Crossmodal identification , 1998, Trends in Cognitive Sciences.

[5]  D. Alais,et al.  Rapid Recalibration to Audiovisual Asynchrony , 2013, The Journal of Neuroscience.

[6]  M. Nitsche,et al.  Excitability changes induced in the human motor cortex by weak transcranial direct current stimulation , 2000, The Journal of physiology.

[7]  S. Rossi,et al.  Safety, ethical considerations, and application guidelines for the use of transcranial magnetic stimulation in clinical practice and research , 2009, Clinical Neurophysiology.

[8]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[9]  Cécile Colin,et al.  Top-down and bottom-up modulation of audiovisual integration in speech , 2005 .

[10]  Jordan Grafman,et al.  Recharging cognition with DC brain polarization , 2005, Trends in Cognitive Sciences.

[11]  L. Cohen,et al.  Transcranial direct current stimulation: State of the art 2008 , 2008, Brain Stimulation.

[12]  L. Young,et al.  Survey of eye movement recording methods , 1975 .

[13]  G. Calvert Crossmodal processing in the human brain: insights from functional neuroimaging studies. , 2001, Cerebral cortex.

[14]  Antoinette T. Gesi,et al.  Long-term training, transfer, and retention in learning to lipread , 1993, Perception & psychophysics.

[15]  Walter Paulus,et al.  Transcranial Direct Current Stimulation and Visual Perception , 2008, Perception.

[16]  C. Coelho,et al.  Auditory-visual speech perception in an adult with aphasia , 2004, Brain injury.

[17]  L. Merabet,et al.  Clinical research with transcranial direct current stimulation (tDCS): Challenges and future directions , 2012, Brain Stimulation.

[18]  John J. Foxe,et al.  Seeing voices: High-density electrical mapping and source-analysis of the multisensory mismatch negativity evoked during the McGurk illusion , 2007, Neuropsychologia.

[19]  P. Bertelson,et al.  Recalibration of temporal order perception by exposure to audio-visual asynchrony. , 2004, Brain research. Cognitive brain research.

[20]  John R. Williams DECLARATION OF HELSINKI , 1964 .

[21]  Claus Tempelmann,et al.  Examining the McGurk illusion using high-field 7 Tesla functional MRI , 2012, Front. Hum. Neurosci..

[22]  Dani Byrd,et al.  Auditory Selective Attention: An fMRI Investigation , 1996, NeuroImage.

[23]  Jeremy I. Skipper,et al.  Seeing Voices : How Cortical Areas Supporting Speech Production Mediate Audiovisual Speech Perception , 2007 .

[24]  N. Bolognini,et al.  Interactions between Senses: Updating on Neural Mechanisms and Behavioral Evidence , 2012, Front. Psychology.

[25]  M. Sams,et al.  Attention to visual speech gestures enhances hemodynamic activity in the left planum temporale , 2006, Human brain mapping.

[26]  R. Hari,et al.  Seeing speech: visual information from lip movements modifies activity in the human auditory cortex , 1991, Neuroscience Letters.

[27]  A. Liberman,et al.  The motor theory of speech perception revised , 1985, Cognition.

[28]  G. Rizzolatti,et al.  Speech listening specifically modulates the excitability of tongue muscles: a TMS study , 2002, The European journal of neuroscience.

[29]  N. Bolognini,et al.  Brain polarization of parietal cortex augments training-induced improvement of visual exploratory and attentional skills , 2010, Brain Research.

[30]  N. Bolognini,et al.  Enhancing multisensory spatial orienting by brain polarization of the parietal cortex , 2010, The European journal of neuroscience.

[31]  M. Funnell,et al.  Hemispheric contributions to the integration of visual and auditory information in speech perception , 1994, Perception & psychophysics.

[32]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[33]  Y. Sugita,et al.  Auditory-visual speech perception examined by fMRI and PET , 2003, Neuroscience Research.

[34]  T. Ahonen,et al.  Audiovisual speech perception in children with developmental language disorder in degraded listening conditions. , 2013, Journal of speech, language, and hearing research : JSLHR.

[35]  E Macaluso,et al.  Spatial and temporal factors during processing of audiovisual speech: a PET study , 2004, NeuroImage.

[36]  Agnes Volein,et al.  Electrophysiological evidence of illusory audiovisual speech percept in human infants , 2008, Proceedings of the National Academy of Sciences.

[37]  Nadia Bolognini,et al.  Neuromodulation of Early Multisensory Interactions in the Visual Cortex , 2013, Journal of Cognitive Neuroscience.

[38]  S. Nishida,et al.  Recalibration of audiovisual simultaneity , 2004, Nature Neuroscience.

[39]  D. Knill,et al.  The Bayesian brain: the role of uncertainty in neural coding and computation , 2004, Trends in Neurosciences.

[40]  Nadia Bolognini,et al.  Review Article , 2003 .

[41]  J. Pineda,et al.  Motor-auditory-visual integration: The role of the human mirror neuron system in communication and communication disorders. , 2009, Journal of communication disorders.

[42]  L. Merabet,et al.  Multisensory processing after a brain damage: Clues on post-injury crossmodal plasticity from neuropsychology , 2013, Neuroscience & Biobehavioral Reviews.

[43]  D. Burnham,et al.  Perception of the auditory‐visual illusion in speech perception by children with phonological disorders , 2008, Clinical linguistics & phonetics.

[44]  Elizabeth Milne,et al.  A Comparison of the Development of Audiovisual Integration in Children with Autism Spectrum Disorders and Typically Developing Children , 2010, Journal of autism and developmental disorders.

[45]  Nadia Bolognini,et al.  Behavioural facilitation following brain stimulation: Implications for neurorehabilitation , 2011, Neuropsychological rehabilitation.

[46]  Nadia Bolognini,et al.  Neuromodulation of multisensory perception: A tDCS study of the sound-induced flash illusion , 2011, Neuropsychologia.

[47]  F. Fregni,et al.  Bilateral temporal cortex transcranial direct current stimulation worsens male performance in a multisensory integration task , 2012, Neuroscience Letters.

[48]  Michael S. Beauchamp,et al.  A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion , 2012, NeuroImage.

[49]  Jean Vroomen,et al.  Brain activation during audiovisual exposure anticipates future perception of ambiguous speech , 2011, NeuroImage.

[50]  Audrey R. Nath,et al.  fMRI-Guided Transcranial Magnetic Stimulation Reveals That the Superior Temporal Sulcus Is a Cortical Locus of the McGurk Effect , 2010, The Journal of Neuroscience.

[51]  G. Sartori,et al.  Lie-specific involvement of dorsolateral prefrontal cortex in deception. , 2008, Cerebral cortex.

[52]  J. Driver,et al.  Multisensory Interplay Reveals Crossmodal Influences on ‘Sensory-Specific’ Brain Regions, Neural Responses, and Judgments , 2008, Neuron.

[53]  Mikko Sams,et al.  Audio–visual speech perception is special , 2005, Cognition.

[54]  B. Gandevia,et al.  DECLARATION OF HELSINKI. , 1964, The Medical journal of Australia.

[55]  M. Hallett,et al.  Neural Correlates of Auditory–Visual Stimulus Onset Asynchrony Detection , 2001, The Journal of Neuroscience.

[56]  C. Sinke,et al.  Disinhibited feedback as a cause of synesthesia: Evidence from a functional connectivity study on auditory-visual synesthetes , 2012, Neuropsychologia.