Neural oscillations in the temporal pole for a temporally congruent audio-visual speech detection task

Though recent studies have elucidated the earliest mechanisms of processing in multisensory integration, our understanding of how multisensory integration of more sustained and complicated stimuli is implemented in higher-level association cortices is lacking. In this study, we used magnetoencephalography (MEG) to determine how neural oscillations alter local and global connectivity during multisensory integration processing. We acquired MEG data from 15 healthy volunteers performing an audio-visual speech matching task. We selected regions of interest (ROIs) using whole brain time-frequency analyses (power spectrum density and wavelet transform), then applied phase amplitude coupling (PAC) and imaginary coherence measurements to them. We identified prominent delta band power in the temporal pole (TP), and a remarkable PAC between delta band phase and beta band amplitude. Furthermore, imaginary coherence analysis demonstrated that the temporal pole and well-known multisensory areas (e.g., posterior parietal cortex and post-central areas) are coordinated through delta-phase coherence. Thus, our results suggest that modulation of connectivity within the local network, and of that between the local and global network, is important for audio-visual speech integration. In short, these neural oscillatory mechanisms within and between higher-level association cortices provide new insights into the brain mechanism underlying audio-visual integration.

[1]  D. Poeppel,et al.  Phase Patterns of Neuronal Responses Reliably Discriminate Speech in Human Auditory Cortex , 2007, Neuron.

[2]  Song-Lin Ding,et al.  Parcellation of human temporal polar cortex: A combined analysis of multiple cytoarchitectonic, chemoarchitectonic, and pathological markers , 2009, The Journal of comparative neurology.

[3]  Oded Ghitza,et al.  Linking Speech Perception and Neurophysiology: Speech Decoding Guided by Cascaded Oscillators Locked to the Input Rhythm , 2011, Front. Psychology.

[4]  L. Pylkkänen,et al.  Basic linguistic composition recruits the left anterior temporal lobe and left angular gyrus during both listening and reading. , 2013, Cerebral cortex.

[5]  W. Singer,et al.  Modulation of Neuronal Interactions Through Neuronal Synchronization , 2007, Science.

[6]  Daniel Tranel,et al.  Investigating the Anterior Temporal Lobe With Direct Intracranial Recordings. , 2015, Neurosurgery.

[7]  P. Gribble,et al.  Temporal constraints on the McGurk effect , 1996, Perception & psychophysics.

[8]  Martin Vetterli,et al.  Wavelets and filter banks: theory and design , 1992, IEEE Trans. Signal Process..

[9]  Liina Pylkkänen,et al.  The role of the left anterior temporal lobe in semantic composition vs. semantic memory , 2014, Neuropsychologia.

[10]  J. Simon,et al.  Neural coding of continuous speech in auditory cortex during monaural and dichotic listening. , 2012, Journal of neurophysiology.

[11]  P. Fries A mechanism for cognitive dynamics: neuronal communication through neuronal coherence , 2005, Trends in Cognitive Sciences.

[12]  C. Schroeder,et al.  Low-frequency neuronal oscillations as instruments of sensory selection , 2009, Trends in Neurosciences.

[13]  J. M. Moran,et al.  Local and long-range functional connectivity is reduced in concert in autism spectrum disorders , 2013, Proceedings of the National Academy of Sciences.

[14]  Asif A Ghazanfar,et al.  Dynamic faces speed up the onset of auditory cortical spiking responses during vocal detection , 2013, Proceedings of the National Academy of Sciences.

[15]  E. Liebenthal,et al.  Neural pathways for visual speech perception , 2014, Front. Neurosci..

[16]  Adriano B. L. Tort,et al.  Dynamic cross-frequency couplings of local field potential oscillations in rat striatum and hippocampus during performance of a T-maze task , 2008, Proceedings of the National Academy of Sciences.

[17]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[18]  Manuel R. Mercier,et al.  Auditory-driven phase reset in visual cortex: Human electrocorticography reveals mechanisms of early multisensory integration , 2013, NeuroImage.

[19]  Denis Cousineau,et al.  How to use MATLAB to fit the ex-Gaussian and other probability functions to a distribution of response times , 2008 .

[20]  A. Pérez-Villalba Rhythms of the Brain, G. Buzsáki. Oxford University Press, Madison Avenue, New York (2006), Price: GB £42.00, p. 448, ISBN: 0-19-530106-4 , 2008 .

[21]  Marie Frei,et al.  Phonetic Data Analysis An Introduction To Fieldwork And Instrumental Techniques , 2016 .

[22]  S. Nishida,et al.  Recalibration of audiovisual simultaneity , 2004, Nature Neuroscience.

[23]  R. Ilmoniemi,et al.  Interpreting magnetic fields of the brain: minimum norm estimates , 2006, Medical and Biological Engineering and Computing.

[24]  E. DeYoe,et al.  A comparison of visual and auditory motion processing in human cerebral cortex. , 2000, Cerebral cortex.

[25]  David Poeppel,et al.  The analysis of speech in different temporal integration windows: cerebral lateralization as 'asymmetric sampling in time' , 2003, Speech Commun..

[26]  S. Scott,et al.  Identification of a pathway for intelligible speech in the left temporal lobe. , 2000, Brain : a journal of neurology.

[27]  John J. Foxe,et al.  Neuro-Oscillatory Phase Alignment Drives Speeded Multisensory Response Times: An Electro-Corticographic Investigation , 2015, The Journal of Neuroscience.

[28]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[29]  David Poeppel,et al.  Cortical oscillations and speech processing: emerging computational principles and operations , 2012, Nature Neuroscience.

[30]  Matthew A. Lambon Ralph,et al.  Convergent Connectivity and Graded Specialization in the Rostral Human Temporal Lobe as Revealed by Diffusion-Weighted Imaging Probabilistic Tractography , 2012, Journal of Cognitive Neuroscience.

[31]  Lizabeth M Romanski,et al.  Representation and integration of auditory and visual stimuli in the primate ventral lateral prefrontal cortex. , 2007, Cerebral cortex.

[32]  A. Ghazanfar,et al.  Is neocortex essentially multisensory? , 2006, Trends in Cognitive Sciences.

[33]  J. Driver,et al.  Multisensory Interplay Reveals Crossmodal Influences on ‘Sensory-Specific’ Brain Regions, Neural Responses, and Judgments , 2008, Neuron.

[34]  John J. Foxe,et al.  Look who's talking: The deployment of visuo-spatial attention during multisensory speech processing under noisy environmental conditions , 2008, NeuroImage.

[35]  R. Traub,et al.  A mechanism for generation of long-range synchronous fast oscillations in the cortex , 1996, Nature.

[36]  S. Taulu,et al.  Suppression of Interference and Artifacts by the Signal Space Separation Method , 2003, Brain Topography.

[37]  D. Pisoni,et al.  Auditory-visual speech perception and synchrony detection for speech and nonspeech signals. , 2006, The Journal of the Acoustical Society of America.

[38]  L. Hazrati,et al.  In vitro recordings of human neocortical oscillations. , 2015, Cerebral cortex.

[39]  Anders M. Dale,et al.  Automatic parcellation of human cortical gyri and sulci using standard anatomical nomenclature , 2010, NeuroImage.

[40]  C. Spence,et al.  Audiovisual synchrony perception for music, speech, and object actions , 2006, Brain Research.

[41]  Gregory B. Cogan,et al.  Visual Input Enhances Selective Speech Envelope Tracking in Auditory Cortex at a “Cocktail Party” , 2013, The Journal of Neuroscience.

[42]  D. V. van Essen,et al.  Corticocortical connections of visual, sensorimotor, and multimodal processing areas in the parietal lobe of the macaque monkey , 2000, The Journal of comparative neurology.

[43]  I. Olson,et al.  The Enigmatic temporal pole: a review of findings on social and emotional processing. , 2007, Brain : a journal of neurology.

[44]  John J. Foxe,et al.  Crossmodal binding through neural coherence: implications for multisensory processing , 2008, Trends in Neurosciences.

[45]  T. Rogers,et al.  Where do you know what you know? The representation of semantic knowledge in the human brain , 2007, Nature Reviews Neuroscience.

[46]  T. Hafting,et al.  Frequency of gamma oscillations routes flow of information in the hippocampus , 2009, Nature.

[47]  O. Bertrand,et al.  Visual Activation and Audiovisual Interactions in the Auditory Cortex during Speech Perception: Intracranial Recordings in Humans , 2008, The Journal of Neuroscience.

[48]  A. Treisman,et al.  Parietal contributions to visual feature binding: evidence from a patient with bilateral lesions , 1995, Science.

[49]  Keith Johnson,et al.  Phonetic Feature Encoding in Human Superior Temporal Gyrus , 2014, Science.

[50]  J. Csicsvari,et al.  Organization of cell assemblies in the hippocampus , 2003, Nature.

[51]  Sharon Zmigrod,et al.  The role of the parietal cortex in multisensory and response integration: evidence from transcranial direct current stimulation (tDCS). , 2014, Multisensory research.

[52]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[53]  J. Martinerie,et al.  The brainweb: Phase synchronization and large-scale integration , 2001, Nature Reviews Neuroscience.

[54]  G. Buzsáki,et al.  Temporal Encoding of Place Sequences by Hippocampal Cell Assemblies , 2006, Neuron.

[55]  R. Knight,et al.  The functional role of cross-frequency coupling , 2010, Trends in Cognitive Sciences.

[56]  G. Buzsáki,et al.  A 4 Hz Oscillation Adaptively Synchronizes Prefrontal, VTA, and Hippocampal Activities , 2011, Neuron.

[57]  D. Poeppel,et al.  Temporal window of integration in auditory-visual speech perception , 2007, Neuropsychologia.

[58]  Bruno B Averbeck,et al.  Integration of Auditory and Visual Communication Information in the Primate Ventrolateral Prefrontal Cortex , 2006, The Journal of Neuroscience.

[59]  Jose M Hurtado,et al.  Statistical method for detection of phase-locking episodes in neural oscillations. , 2004, Journal of neurophysiology.

[60]  Jason B. Mattingley,et al.  Parietal disruption alters audiovisual binding in the sound-induced flash illusion , 2012, NeuroImage.

[61]  A. Puce,et al.  Neuronal oscillations and visual amplification of speech , 2008, Trends in Cognitive Sciences.

[62]  O. Jensen,et al.  Cross-frequency coupling between neuronal oscillations , 2007, Trends in Cognitive Sciences.

[63]  Richard M. Leahy,et al.  Brainstorm: A User-Friendly Application for MEG/EEG Analysis , 2011, Comput. Intell. Neurosci..

[64]  Lynne E. Bernstein,et al.  Auditory speech detection in noise enhanced by lipreading , 2004, Speech Commun..

[65]  D. Poeppel,et al.  Auditory Cortex Tracks Both Auditory and Visual Stimulus Dynamics Using Low-Frequency Neuronal Phase Modulation , 2010, PLoS biology.

[66]  Christoph Kayser,et al.  Who is That? Brain Networks and Mechanisms for Identifying Individuals , 2015, Trends in Cognitive Sciences.

[67]  J. O’Keefe,et al.  Phase relationship between hippocampal place units and the EEG theta rhythm , 1993, Hippocampus.

[68]  M. Hallett,et al.  Identifying true brain interaction from EEG data using the imaginary part of coherency , 2004, Clinical Neurophysiology.

[69]  W. Singer Synchronization of cortical activity and its putative role in information processing and learning. , 1993, Annual review of physiology.

[70]  A. Dale,et al.  Cortical Surface-Based Analysis II: Inflation, Flattening, and a Surface-Based Coordinate System , 1999, NeuroImage.

[71]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[72]  Tianzi Jiang,et al.  Connectivity-based parcellation of the human temporal pole using diffusion tensor imaging. , 2014, Cerebral cortex.

[73]  Pascal Belin,et al.  Dissociating task difficulty from incongruence in face-voice emotion integration , 2013, Front. Hum. Neurosci..

[74]  John O'Keefe,et al.  Independent rate and temporal coding in hippocampal pyramidal cells , 2003, Nature.

[75]  John J. Foxe,et al.  Do you see what I am saying? Exploring visual enhancement of speech comprehension in noisy environments. , 2006, Cerebral cortex.

[76]  Adriano B. L. Tort,et al.  Theta–gamma coupling increases during the learning of item–context associations , 2009, Proceedings of the National Academy of Sciences.

[77]  J. Schwartz,et al.  Seeing to hear better: evidence for early audio-visual interactions in speech identification , 2004, Cognition.

[78]  Christoph Kayser,et al.  Natural asynchronies in audiovisual communication signals regulate neuronal multisensory interactions in voice-sensitive cortex , 2014, Proceedings of the National Academy of Sciences.