The Motor Network Reduces Multisensory Illusory Perception

Observing mouth movements has strikingly effects on the perception of speech. Any mismatch between sound and mouth movements will result in listeners perceiving illusory consonants (McGurk effect), whereas matching mouth movements assist with the correct recognition of speech sounds. Recent neuroimaging studies have yielded evidence that the motor areas are involved in speech processing, yet their contributions to multisensory illusion remain unclear. Using functional magnetic resonance imaging (fMRI) and transcranial magnetic stimulation (TMS) in an event-related design, we aimed to identify the functional roles of the motor network in the occurrence of multisensory illusion in female and male brains. fMRI showed bilateral activation of the inferior frontal gyrus (IFG) in audiovisually incongruent trials. Activity in the left IFG was negatively correlated with occurrence of the McGurk effect. The effective connectivity between the left IFG and the bilateral precentral gyri was stronger in incongruent than in congruent trials. The McGurk effect was reduced in incongruent trials by applying single-pulse TMS to motor cortex (M1) lip areas, indicating that TMS facilitates the left IFG-precentral motor network to reduce the McGurk effect. TMS of the M1 lip areas was effective in reducing the McGurk effect within the specific temporal range from 100 ms before to 200 ms after the auditory onset, and TMS of the M1 foot area did not influence the McGurk effect, suggesting topographical specificity. These results provide direct evidence that the motor network makes specific temporal and topographical contributions to the processing of multisensory integration of speech to avoid illusion. SIGNIFICANCE STATEMENT The human motor network, including the inferior frontal gyrus and primary motor cortex lip area, appears to be involved in speech perception, but the functional contribution to the McGurk effect is unknown. Functional magnetic resonance imaging revealed that activity in these areas of the motor network increased when the audiovisual stimuli were incongruent, and that the increased activity was negatively correlated with perception of the McGurk effect. Furthermore, applying transcranial magnetic stimulation to the motor areas reduced the McGurk effect. These two observations provide evidence that the motor network contributes to the avoidance of multisensory illusory perception.

[1]  F. Lin,et al.  Primary and multisensory cortical activity is correlated with audiovisual percepts , 2009, Human brain mapping.

[2]  Audrey R. Nath,et al.  fMRI-Guided Transcranial Magnetic Stimulation Reveals That the Superior Temporal Sulcus Is a Cortical Locus of the McGurk Effect , 2010, The Journal of Neuroscience.

[3]  K. Watkins,et al.  Motor Representations of Articulators Contribute to Categorical Perception of Speech Sounds , 2009, The Journal of Neuroscience.

[4]  Rainer Goebel,et al.  Optimizing Functional Accuracy of TMS in Cognitive Studies: A Comparison of Methods , 2009, Journal of Cognitive Neuroscience.

[5]  Jeffery A. Jones,et al.  Phonetic perceptual identification by native- and second-language speakers differentially activates brain regions involved with acoustic phonetic processing and those involved with articulatory–auditory/orosensory internal models , 2004, NeuroImage.

[6]  U. Ziemann,et al.  Left dorsal speech stream components and their contribution to phonological processing , 2015, Brain Stimulation.

[7]  糺 北原,et al.  骨導超音波の temporal window of integration について , 2014 .

[8]  M. Hallett,et al.  Neural Correlates of Auditory–Visual Stimulus Onset Asynchrony Detection , 2001, The Journal of Neuroscience.

[9]  J. Trueswell,et al.  Cognitive control and parsing: Reexamining the role of Broca’s area in sentence comprehension , 2005, Cognitive, affective & behavioral neuroscience.

[10]  L. Fadiga,et al.  The Motor Somatotopy of Speech Perception , 2009, Current Biology.

[11]  D. Poeppel,et al.  Temporal window of integration in auditory-visual speech perception , 2007, Neuropsychologia.

[12]  M. Arbib,et al.  Language within our grasp , 1998, Trends in Neurosciences.

[13]  Mikko Sams,et al.  Processing of audiovisual speech in Broca's area , 2005, NeuroImage.

[14]  Gregory Hickok,et al.  Audiovisual Speech Integration Does Not Rely on the Motor System: Evidence from Articulatory Suppression, the McGurk Effect, and fMRI , 2014, Journal of Cognitive Neuroscience.

[15]  T. Paus,et al.  Modulation of Motor Excitability during Speech Perception: The Role of Broca's Area , 2004, Journal of Cognitive Neuroscience.

[16]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[17]  D. Pandya,et al.  Afferent cortical connections and architectonics of the superior temporal sulcus and surrounding cortex in the rhesus monkey , 1978, Brain Research.

[18]  T. Paus,et al.  Seeing and hearing speech excites the motor system involved in speech production , 2003, Neuropsychologia.

[19]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[20]  Jeremy I. Skipper,et al.  Seeing Voices : How Cortical Areas Supporting Speech Production Mediate Audiovisual Speech Perception , 2007 .

[21]  U. Ziemann,et al.  Effective connectivity hierarchically links temporoparietal and frontal areas of the auditory dorsal stream with the motor cortex lip area during speech perception , 2012, Brain and Language.

[22]  R. Sekuler,et al.  Sound alters visual motion perception , 1997, Nature.

[23]  M. Wallace,et al.  Individual differences in the multisensory temporal binding window predict susceptibility to audiovisual illusions. , 2012, Journal of experimental psychology. Human perception and performance.

[24]  Karl J. Friston,et al.  Psychophysiological and Modulatory Interactions in Neuroimaging , 1997, NeuroImage.

[25]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[26]  G. Rizzolatti,et al.  The mirror-neuron system. , 2004, Annual review of neuroscience.

[27]  G. Rizzolatti,et al.  Speech listening specifically modulates the excitability of tongue muscles: a TMS study , 2002, The European journal of neuroscience.

[28]  U. Ziemann,et al.  Observation-execution matching and action inhibition in human primary motor cortex during viewing of speech-related lip movements or listening to speech , 2011, Neuropsychologia.

[29]  C. Price,et al.  The role of the posterior superior temporal sulcus in audiovisual processing. , 2008, Cerebral cortex.

[30]  Marco Iacoboni,et al.  Lateralization in motor facilitation during action observation: a TMS study , 2001, NeuroImage.

[31]  U. Ziemann,et al.  Utility of TMS to understand the neurobiology of speech , 2013, Front. Psychol..

[32]  A. Liberman,et al.  The motor theory of speech perception revised , 1985, Cognition.

[33]  R. Hari,et al.  Viewing Lip Forms Cortical Dynamics , 2002, Neuron.

[34]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[35]  Luigi Cattaneo,et al.  On the tip of the tongue: Modulation of the primary motor cortex during audiovisual speech perception , 2010, Speech Commun..

[36]  Robert Chen,et al.  Observation–execution matching system for speech: a magnetic stimulation study , 2001, Neuroreport.

[37]  M. Carrasco,et al.  Occipital Transcranial Magnetic Stimulation Has an Activity-Dependent Suppressive Effect , 2012, The Journal of Neuroscience.

[38]  Michael S. Beauchamp,et al.  A neural basis for interindividual differences in the McGurk effect, a multisensory speech illusion , 2012, NeuroImage.

[39]  M. Corballis Mirror neurons and the evolution of language , 2010, Brain and Language.

[40]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[41]  Geraint Rees,et al.  Stochastic Resonance Effects Reveal the Neural Mechanisms of Transcranial Magnetic Stimulation , 2011, The Journal of Neuroscience.

[42]  Steven L. Small,et al.  Listening to talking faces: motor cortical activation during speech perception , 2005, NeuroImage.

[43]  R. Ilmoniemi,et al.  Functional links between motor and language systems , 2005, The European journal of neuroscience.

[44]  D. Angelaki,et al.  Multisensory integration: resolving sensory ambiguities to build novel representations , 2010, Current Opinion in Neurobiology.

[45]  Lee M. Miller,et al.  Behavioral/systems/cognitive Perceptual Fusion and Stimulus Coincidence in the Cross- Modal Integration of Speech , 2022 .