Visual face-movement sensitive cortex is relevant for auditory-only speech recognition

It is commonly assumed that the recruitment of visual areas during audition is not relevant for performing auditory tasks ('auditory-only view'). According to an alternative view, however, the recruitment of visual cortices is thought to optimize auditory-only task performance ('auditory-visual view'). This alternative view is based on functional magnetic resonance imaging (fMRI) studies. These studies have shown, for example, that even if there is only auditory input available, face-movement sensitive areas within the posterior superior temporal sulcus (pSTS) are involved in understanding what is said (auditory-only speech recognition). This is particularly the case when speakers are known audio-visually, that is, after brief voice-face learning. Here we tested whether the left pSTS involvement is causally related to performance in auditory-only speech recognition when speakers are known by face. To test this hypothesis, we applied cathodal transcranial direct current stimulation (tDCS) to the pSTS during (i) visual-only speech recognition of a speaker known only visually to participants and (ii) auditory-only speech recognition of speakers they learned by voice and face. We defined the cathode as active electrode to down-regulate cortical excitability by hyperpolarization of neurons. tDCS to the pSTS interfered with visual-only speech recognition performance compared to a control group without pSTS stimulation (tDCS to BA6/44 or sham). Critically, compared to controls, pSTS stimulation additionally decreased auditory-only speech recognition performance selectively for voice-face learned speakers. These results are important in two ways. First, they provide direct evidence that the pSTS is causally involved in visual-only speech recognition; this confirms a long-standing prediction of current face-processing models. Secondly, they show that visual face-sensitive pSTS is causally involved in optimizing auditory-only speech recognition. These results are in line with the 'auditory-visual view' of auditory speech perception, which assumes that auditory speech recognition is optimized by using predictions from previously encoded speaker-specific audio-visual internal models.

[1]  J. Rothwell,et al.  Speech Facilitation by Left Inferior Frontal Cortex Stimulation , 2011, Current Biology.

[2]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[3]  S. Rossi,et al.  Safety, ethical considerations, and application guidelines for the use of transcranial magnetic stimulation in clinical practice and research , 2009, Clinical Neurophysiology.

[4]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[5]  B. Argall,et al.  Unraveling multisensory integration: patchy organization within human STS multisensory cortex , 2004, Nature Neuroscience.

[6]  Audrey R. Nath,et al.  fMRI-Guided Transcranial Magnetic Stimulation Reveals That the Superior Temporal Sulcus Is a Cortical Locus of the McGurk Effect , 2010, The Journal of Neuroscience.

[7]  Michael S. Beauchamp,et al.  Multisensory speech perception without the left superior temporal sulcus , 2012, NeuroImage.

[8]  L. Rosenblum,et al.  Lip-Read Me Now, Hear Me Better Later , 2006, Psychological science.

[9]  Nadia Bolognini,et al.  Tuning and disrupting the brain—modulating the McGurk illusion with electrical stimulation , 2014, Front. Hum. Neurosci..

[10]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[11]  P. McGuire,et al.  Cortical substrates for the perception of face actions: an fMRI study of the specificity of activation for seen speech and for meaningless lower-face acts (gurning). , 2001, Brain research. Cognitive brain research.

[12]  Anne-Lise Giraud,et al.  Voice recognition and cross-modal responses to familiar speakers' voices in prosopagnosia. , 2006, Cerebral cortex.

[13]  Gregory Hickok,et al.  An fMRI Study of Audiovisual Speech Perception Reveals Multisensory Interactions in Auditory Cortex , 2013, PloS one.

[14]  Luc H. Arnal,et al.  Dual Neural Routing of Visual Facilitation in Speech Processing , 2009, The Journal of Neuroscience.

[15]  William M. Stern,et al.  Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex , 2007, Nature Neuroscience.

[16]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[17]  Gereon R. Fink,et al.  Enhancing language performance with non-invasive brain stimulation—A transcranial direct current stimulation study in healthy humans , 2008, Neuropsychologia.

[18]  Andreas Kleinschmidt,et al.  Interaction of Face and Voice Areas during Speaker Recognition , 2005, Journal of Cognitive Neuroscience.

[19]  Sachiko Koyama,et al.  Comprehension of degraded speech sounds with m-sequence modulation: An fMRI study , 2010, NeuroImage.

[20]  V. Mann,et al.  Influence of vocalic context on perception of the [∫]-[s] distinction , 1978 .

[21]  Cheryl M. Capek,et al.  Superior temporal activation as a function of linguistic knowledge: Insights from deaf native signers who speechread , 2010, Brain and Language.

[22]  Cheryl M. Capek,et al.  Cortical circuits for silent speechreading in deaf and hearing people , 2008, Neuropsychologia.

[23]  Ladan Shams,et al.  Early modulation of visual cortex by sound: an MEG study , 2005, Neuroscience Letters.

[24]  Patrik Vuilleumier,et al.  Reactivation of visual cortex during memory retrieval: Content specificity and emotional modulation , 2012, NeuroImage.

[25]  Uta Noppeney,et al.  Temporal prediction errors in visual and auditory cortices , 2014, Current Biology.

[26]  T. Allison,et al.  Functional anatomy of biological motion perception in posterior temporal cortex: an FMRI study of eye, mouth and hand movements. , 2005, Cerebral cortex.

[27]  T Allison,et al.  ERPS EVOKED BY VIEWING FACIAL MOVEMENTS , 2000, Cognitive neuropsychology.

[28]  A. Giraud,et al.  Implicit Multisensory Associations Influence Voice Recognition , 2006, PLoS biology.

[29]  R. Hari,et al.  Viewing Lip Forms Cortical Dynamics , 2002, Neuron.

[30]  J. Delgado-García,et al.  Transcranial direct-current stimulation modulates synaptic mechanisms involved in associative learning in behaving rabbits , 2012, Proceedings of the National Academy of Sciences.

[31]  K. Kriegstein,et al.  Visual abilities are important for auditory-only speech recognition: Evidence from autism spectrum disorder , 2014, Neuropsychologia.

[32]  W. H. Sumby,et al.  Visual contribution to speech intelligibility in noise , 1954 .

[33]  L. Cohen,et al.  Transcranial direct current stimulation: State of the art 2008 , 2008, Brain Stimulation.

[34]  David Poeppel,et al.  Feedforward and feedback in speech perception: Revisiting analysis by synthesis , 2011 .

[35]  L. Merabet,et al.  Occipital Transcranial Magnetic Stimulation Has Opposing Effects on Visual and Auditory Stimulus Detection: Implications for Multisensory Interactions , 2007, The Journal of Neuroscience.

[36]  M. Nitsche,et al.  Excitability changes induced in the human motor cortex by weak transcranial direct current stimulation , 2000, The Journal of physiology.

[37]  Paolo Maria Rossini,et al.  Naming facilitation induced by transcranial direct current stimulation , 2010, Behavioural Brain Research.

[38]  S. Kosslyn,et al.  Visual Mental Imagery Activates Topographically Organized Visual Cortex: PET Investigations , 1993, Journal of Cognitive Neuroscience.

[39]  K. von Kriegstein,et al.  Functional Connectivity between Face-Movement and Speech-Intelligibility Areas during Auditory-Only Speech Perception , 2014, PloS one.

[40]  Karl J. Friston.  The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[41]  D. Pitcher,et al.  Transcranial Magnetic Stimulation Disrupts the Perception and Embodiment of Facial Expressions , 2008, The Journal of Neuroscience.

[42]  G. E. Peterson,et al.  Control Methods Used in a Study of the Vowels , 1951 .

[43]  Ethan R. Buch,et al.  Noninvasive cortical stimulation enhances motor skill acquisition over multiple days through an effect on consolidation , 2009, Proceedings of the National Academy of Sciences.

[44]  M J Brammer,et al.  Dispersed activation in the left temporal cortex for speech-reading in congenitally deaf people , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[45]  R. Campbell,et al.  Reading Speech from Still and Moving Faces: The Neural Substrates of Visible Speech , 2003, Journal of Cognitive Neuroscience.

[46]  Antoinette T. Gesi,et al.  Long-term training, transfer, and retention in learning to lipread , 1993, Perception & psychophysics.

[47]  Mark Hallett,et al.  Repetitive transcranial magnetic stimulation or transcranial direct current stimulation? , 2009, Brain Stimulation.

[48]  David Poeppel,et al.  Visual speech speeds up the neural processing of auditory speech. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[49]  T. Allison,et al.  Temporal Cortex Activation in Humans Viewing Eye and Mouth Movements , 1998, The Journal of Neuroscience.

[50]  Katharina von Kriegstein,et al.  Mechanisms of enhancing visual–speech recognition by prior auditory information , 2013, NeuroImage.

[51]  Ryan A. Stevenson,et al.  Audiovisual integration in human superior temporal sulcus: Inverse effectiveness and the neural processing of speech and object recognition , 2009, NeuroImage.

[52]  Sergio P. Rigonatti,et al.  Enhancement of non-dominant hand motor function by anodal transcranial direct current stimulation , 2006, Neuroscience Letters.

[53]  Robert T. Knight,et al.  Superior Temporal SulcusIt's My Area: Or Is It? , 2008, Journal of Cognitive Neuroscience.

[54]  Alfred Anwander,et al.  Direct Structural Connections between Voice- and Face-Recognition Areas , 2011, The Journal of Neuroscience.

[55]  Deborah A. Hall,et al.  Reading Fluent Speech from Talking Faces: Typical Brain Networks and Individual Differences , 2005, Journal of Cognitive Neuroscience.

[56]  K. Sims,et al.  Diseases of adenosine triphosphate synthesis in children , 2002, Current opinion in neurology.

[57]  Karen Lander,et al.  Does face familiarity influence speechreadability? , 2008, Quarterly journal of experimental psychology.

[58]  Karl J. Friston,et al.  Recognizing Sequences of Sequences , 2009, PLoS Comput. Biol..

[59]  H. Kimmel,et al.  Three criteria for the use of one-tailed tests. , 1957, Psychological bulletin.

[60]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[61]  A. Ghazanfar,et al.  Is neocortex essentially multisensory? , 2006, Trends in Cognitive Sciences.

[62]  R. C. Oldfield.  The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[63]  J. Haxby,et al.  The distributed human neural system for face perception , 2000, Trends in Cognitive Sciences.

[64]  S. Scott,et al.  Identification of a pathway for intelligible speech in the left temporal lobe. , 2000, Brain : a journal of neurology.

[65]  L. Jancke,et al.  Brain stimulation modulates driving behavior , 2015 .

[66]  J M Bland,et al.  Statistics Notes: One and two sided tests of significance , 1994 .

[67]  N. Kanwisher,et al.  Mental Imagery of Faces and Places Activates Corresponding Stimulus-Specific Brain Regions , 2000, Journal of Cognitive Neuroscience.

[68]  M. Nitsche,et al.  Pharmacological approach to the mechanisms of transcranial DC-stimulation-induced after-effects of human motor cortex excitability. , 2002, Brain : a journal of neurology.

[69]  Aina Puce,et al.  Common and distinct brain activation to viewing dynamic sequences of face and hand movements , 2007, NeuroImage.

[70]  M. Nitsche,et al.  Excitability changes induced in the human primary visual cortex by transcranial direct current stimulation: direct electrophysiological evidence. , 2004, Investigative ophthalmology & visual science.

[71]  Moshe Bar,et al.  The proactive brain: memory for predictions , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[72]  T. Robbins,et al.  Differential effects of insular and ventromedial prefrontal cortex lesions on risky decision-making , 2008, Brain : a journal of neurology.

[73]  D. Poeppel,et al.  The Human Auditory Cortex , 2012, Springer Handbook of Auditory Research.

[74]  Hanna Damasio,et al.  Predicting visual stimuli on the basis of activity in auditory cortices , 2010, Nature Neuroscience.

[75]  Á. Pascual-Leone,et al.  Repetitive TMS over posterior STS disrupts perception of biological motion , 2005, Vision Research.

[76]  C. Price,et al.  The role of the posterior superior temporal sulcus in audiovisual processing. , 2008, Cerebral cortex.

[77]  Karl J. Friston,et al.  Action and behavior: a free-energy formulation , 2010, Biological Cybernetics.

[78]  Stefan J. Kiebel,et al.  Early auditory sensory processing of voices is facilitated by visual mechanisms , 2013, NeuroImage.

[79]  N. Birbaumer,et al.  Enhancement of Planning Ability by Transcranial Direct Current Stimulation , 2009, The Journal of Neuroscience.

[80]  Stefan J. Kiebel,et al.  Simulation of talking faces in the human brain improves auditory speech recognition , 2008, Proceedings of the National Academy of Sciences.

[81]  John J. Foxe,et al.  Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. , 2006, Cerebral cortex.

[82]  Ruth Campbell,et al.  The processing of audio-visual speech: empirical and neural bases , 2008, Philosophical Transactions of the Royal Society B: Biological Sciences.

[83]  E. Bullmore,et al.  Activation of auditory cortex during silent lipreading. , 1997, Science.

[84]  Kayoko Okada,et al.  Two cortical mechanisms support the integration of visual and auditory speech: A hypothesis and preliminary data , 2009, Neuroscience Letters.

[85]  K. Kriegstein,et al.  Contact dermatitis after transcranial direct current stimulation , 2012, Brain Stimulation.

[86]  M. Sams,et al.  Primary auditory cortex activation by visual speech: an fMRI study at 3 T , 2005, Neuroreport.

[87]  Hans-Jochen Heinze,et al.  Scanning silence: Mental imagery of complex sounds , 2005, NeuroImage.

[88]  O. Bertrand,et al.  Visual Activation and Audiovisual Interactions in the Auditory Cortex during Speech Perception: Intracranial Recordings in Humans , 2008, The Journal of Neuroscience.

[89]  R. Campbell,et al.  Evidence from functional magnetic resonance imaging of crossmodal binding in the human heteromodal cortex , 2000, Current Biology.

[90]  S. M. Sheffert,et al.  Audiovisual speech facilitates voice learning , 2004, Perception & psychophysics.

[91]  Rutvik H. Desai,et al.  Specialization along the Left Superior Temporal Sulcus for Auditory Categorization , 2010, Cerebral cortex.

[92]  Michael S Beauchamp,et al.  See me, hear me, touch me: multisensory integration in lateral occipital-temporal cortex , 2005, Current Opinion in Neurobiology.