Mediofrontal theta‐band oscillations reflect top‐down influence in the ventriloquist illusion

In the ventriloquist illusion, spatially disparate visual signals can influence the perceived location of simultaneous sounds. Previous studies have shown asymmetrical responses in auditory cortical regions following perceived peripheral sound shifts. Moreover, higher‐order cortical areas perform inferences on the sources of disparate audiovisual signals. Recent studies have also highlighted top‐down influence in the ventriloquist illusion and postulated a governing function of neural oscillations for crossmodal processing. In this EEG study, we analyzed source‐reconstructed neural oscillations to address the question of whether perceived sound shifts affect the laterality of auditory responses. Moreover, we investigated the modulation of neural oscillations related to the occurrence of the illusion more generally. With respect to the first question, we did not find evidence for significant changes in the laterality of auditory responses due to perceived sound shifts. However, we found a sustained reduction of mediofrontal theta‐band power starting prior to stimulus onset when participants perceived the illusion compared to when they did not perceive the illusion. We suggest that this effect reflects a state of diminished cognitive control, leading to reliance on more readily discriminable visual information and increased crossmodal influence. We conclude that mediofrontal theta‐band oscillations serve as a neural mechanism underlying top‐down modulation of crossmodal processing in the ventriloquist illusion.

[1]  Robert Oostenveld,et al.  FieldTrip: Open Source Software for Advanced Analysis of MEG, EEG, and Invasive Electrophysiological Data , 2010, Comput. Intell. Neurosci..

[2]  Hiroshi Ando,et al.  Neural correlates of sound externalization , 2013, NeuroImage.

[3]  U. Noppeney,et al.  Cortical Hierarchies Perform Bayesian Causal Inference in Multisensory Perception , 2015, PLoS biology.

[4]  A. von Stein,et al.  Different frequencies for different scales of cortical integration: from local gamma to long range alpha/theta synchronization. , 2000, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[5]  M. Frank,et al.  Frontal theta as a mechanism for cognitive control , 2014, Trends in Cognitive Sciences.

[6]  J. Rauschecker,et al.  Mechanisms and streams for processing of "what" and "where" in auditory cortex. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[7]  Nikos K. Logothetis,et al.  Widespread and Opponent fMRI Signals Represent Sound Location in Macaque Auditory Cortex , 2017, Neuron.

[8]  J. Vroomen,et al.  Intersensory binding across space and time: A tutorial review , 2013, Attention, Perception, & Psychophysics.

[9]  D. Foster,et al.  Model-free estimation of the psychometric function , 2009, Attention, perception & psychophysics.

[10]  C. Grady,et al.  “What” and “where” in the human auditory system , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[11]  P. Bruns.  The Ventriloquist Illusion as a Tool to Study Multisensory Processing: An Update , 2019, Front. Integr. Neurosci..

[12]  E. Wagenmakers,et al.  A default Bayesian hypothesis test for correlations and partial correlations , 2012, Psychonomic bulletin & review.

[13]  Uta Noppeney,et al.  To integrate or not to integrate: Temporal dynamics of Bayesian Causal Inference , 2018, bioRxiv.

[14]  E. Miller,et al.  The prefontral cortex and cognitive control , 2000, Nature Reviews Neuroscience.

[15]  P. Deltenre,et al.  Electrophysiology of spatial scene analysis: the mismatch negativity (MMN) is sensitive to the ventriloquism illusion , 2002, Clinical Neurophysiology.

[16]  S. Hillyard,et al.  Neural Basis of the Ventriloquist Illusion , 2007, Current Biology.

[17]  Terrence J. Sejnowski,et al.  Independent Component Analysis Using an Extended Infomax Algorithm for Mixed Subgaussian and Supergaussian Sources , 1999, Neural Computation.

[18]  B. Rockstroh,et al.  Study of the Human Auditory Cortices Using a Whole-Head Magnetometer: Left vs. Right Hemisphere and Ipsilateral vs. Contralateral Stimulation , 1998, Audiology and Neurotology.

[19]  P. Bertelson,et al.  Cross-modal bias and perceptual fusion with auditory-visual spatial discordance , 1981, Perception & psychophysics.

[20]  Brigitte Röder,et al.  The role of auditory cortex in the spatial ventriloquism aftereffect , 2017, NeuroImage.

[21]  J. Fell,et al.  Lateralized auditory spatial perception and the contralaterality of cortical processing as studied with functional magnetic resonance imaging and magnetoencephalography , 1999, Human brain mapping.

[22]  C. Kayser,et al.  Shared neural underpinnings of multisensory integration and trial-by-trial perceptual recalibration in humans , 2019, eLife.

[23]  M. Murray,et al.  Multisensory Integration: Flexible Use of General Operations , 2014, Neuron.

[24]  Mario Maiworm,et al.  Reward expectation influences audiovisual spatial integration , 2014, Attention, Perception, & Psychophysics.

[25]  Philipp Berens,et al.  CircStat: AMATLABToolbox for Circular Statistics , 2009, Journal of Statistical Software.

[26]  Jean Vroomen,et al.  Illusory sound shifts induced by the ventriloquist illusion evoke the mismatch negativity , 2004, Neuroscience Letters.

[27]  D. Burr,et al.  The Ventriloquist Effect Results from Near-Optimal Bimodal Integration , 2004, Current Biology.

[28]  Uta Noppeney,et al.  Reliability-Weighted Integration of Audiovisual Signals Can Be Modulated by Top-down Attention , 2018, eNeuro.

[29]  N. Weisz,et al.  Prestimulus beta power and phase synchrony influence the sound-induced flash illusion. , 2014, Cerebral cortex.

[30]  Christopher C. Berger,et al.  Mental Imagery Changes Multisensory Perception , 2013, Current Biology.

[31]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[32]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[33]  Christoph Kayser,et al.  Behavioral/systems/cognitive Functional Imaging Reveals Visual Modulation of Specific Fields in Auditory Cortex , 2022 .

[34]  Robert B. Welch,et al.  The “ventriloquist effect”: Visual dominance or response bias? , 1975 .

[35]  Ulrich Pomper,et al.  Distinct patterns of local oscillatory activity and functional connectivity underlie intersensory attention and temporal prediction , 2016, Cortex.

[36]  Dejan Draschkow,et al.  Cluster-based permutation tests of MEG/EEG data do not establish significance of effect latency or location. , 2019, Psychophysiology.

[37]  Mario Maiworm,et al.  When emotional valence modulates audiovisual integration , 2012, Attention, Perception, & Psychophysics.

[38]  M. Scherg,et al.  Intracerebral Sources of Human Auditory-Evoked Potentials , 1999, Audiology and Neurotology.

[39]  N. Weisz,et al.  On the variability of the McGurk effect: audiovisual integration depends on prestimulus brain states. , 2012, Cerebral cortex.

[40]  Théodore Papadopoulo,et al.  OpenMEEG: opensource software for quasistatic bioelectromagnetics , 2010, Biomedical engineering online.

[41]  Brigitte Röder,et al.  Tactile capture of auditory localization: an event‐related potential study , 2010, The European journal of neuroscience.

[42]  W. Drongelen,et al.  Localization of brain electrical activity via linearly constrained minimum variance spatial filtering , 1997, IEEE Transactions on Biomedical Engineering.

[43]  R. Oostenveld,et al.  Nonparametric statistical testing of EEG- and MEG-data , 2007, Journal of Neuroscience Methods.

[44]  Michael X Cohen,et al.  Analyzing Neural Time Series Data: Theory and Practice , 2014 .

[45]  Lee M. Miller,et al.  Populations of auditory cortical neurons can accurately encode acoustic space across stimulus intensity , 2009, Proceedings of the National Academy of Sciences.

[46]  C. Braun,et al.  Prestimulus oscillatory power and connectivity patterns predispose conscious somatosensory perception , 2014, Proceedings of the National Academy of Sciences.

[47]  Uta Noppeney,et al.  Distinct Computational Principles Govern Multisensory Integration in Primary Sensory and Association Cortices , 2016, Current Biology.

[48]  J. Goldberg,et al.  Response of binaural neurons of dog superior olivary complex to dichotic tonal stimuli: some physiological mechanisms of sound localization. , 1969, Journal of neurophysiology.

[49]  Arno Villringer,et al.  Multiple mechanisms link prestimulus neural oscillations to sensory responses , 2018, bioRxiv.

[50]  Michael X. Cohen,et al.  Single-Trial Regression Elucidates the Role of Prefrontal Theta Oscillations in Response Conflict , 2011, Front. Psychology.

[51]  Barbara F. Händel,et al.  Top-Down Controlled Alpha Band Activity in Somatosensory Areas Determines Behavioral Performance in a Discrimination Task , 2011, The Journal of Neuroscience.

[52]  Christopher C. Berger,et al.  The Fusion of Mental Imagery and Sensation in the Temporal Association Cortex , 2014, The Journal of Neuroscience.

[53]  Raphaël V. Meylan,et al.  The spatio-temporal brain dynamics of processing and integrating sound localization cues in humans , 2006, Brain Research.

[54]  Robert Oostenveld,et al.  FieldTrip Made Easy: An Analysis Protocol for Group Analysis of the Auditory Steady State Brain Response in Time, Frequency, and Space , 2018, Front. Neurosci..

[55]  T. Griffiths,et al.  Distinct Mechanisms for Processing Spatial Sequences and Pitch Sequences in the Human Auditory Brain , 2003, The Journal of Neuroscience.

[56]  Julian Keil,et al.  Neural Oscillations Orchestrate Multisensory Processing , 2018, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.