The origin of the audiovisual bounce inducing effect: A TMS study

The audiovisual bounce inducing effect (ABE) is a bouncing percept induced by the presence of a sound at the moment of two moving objects intercepting in a motion display otherwise perceived as streaming. The origin of the ABE is still debated: the effect could arise from the subtraction of attentional resources caused by the sound (needed to favor the perception of streaming), and/or from the cross-modal integration (binding) of visual and auditory information: indeed bouncing-like sounds are best in inducing the ABE. The neural mechanism responsible for the ABE is still unknown. Here, by using offline TMS, we investigated the role of the posterior parietal cortex (PPC), thought to be involved in both attentional and binding processes, in the generation of the ABE. Results show that disrupting the functional integrity of the right (but not the left) PPC has the effect of weakening the binding of cross-modal information, which reduces the magnitude of the ABE. Indeed, if the effect of parietal stimulation was merely to disrupt attention, we would expect an increase (not a decrease) of bouncing percepts. The present study not only shows the involvement of the right PPC in the ABE, but also support the notion that cross-modal binding (and not attention) is at the origin of the ABE.

[1]  Gregor Thut,et al.  Differential effects of low-frequency rTMS at the occipital pole on visual-induced alpha desynchronization and visual-evoked potentials , 2003, NeuroImage.

[2]  G. Pfurtscheller,et al.  Event-related synchronization (ERS) in the alpha band--an electrophysiological correlate of cortical idling: a review. , 1996, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[3]  T Mulholland,et al.  Human EEG, behavioral stillness and biofeedback. , 1995, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[4]  C. Spence,et al.  Crossmodal binding: Evaluating the “unity assumption” using audiovisual speech stimuli , 2007, Perception & psychophysics.

[5]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[6]  Jason B. Mattingley,et al.  Parietal disruption impairs reflexive spatial attention within and between sensory modalities , 2007, Neuropsychologia.

[7]  Clara Casco,et al.  Audiovisual bounce-inducing effect: When sound congruence affects grouping in vision , 2010, Attention, perception & psychophysics.

[8]  G Pfurtscheller,et al.  Event-Related changes of band power and coherence: methodology and interpretation. , 1999, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[9]  George A. Alvarez,et al.  The Role of the Parietal Lobe in Visual Extinction Studied with Transcranial Magnetic Stimulation , 2009, Journal of Cognitive Neuroscience.

[10]  A. Drzezga,et al.  Continuous Transcranial Magnetic Stimulation during Positron Emission Tomography: A Suitable Tool for Imaging Regional Excitability of the Human Cortex , 2001, NeuroImage.

[11]  L. Cohen,et al.  Reduction of human visual cortex excitability using 1-Hz transcranial magnetic stimulation , 2000, Neurology.

[12]  Gianluca Campana,et al.  Priming of motion direction and area V5/MT: a test of perceptual memory. , 2002, Cerebral cortex.

[13]  Debora Brignani,et al.  Modulation of cortical oscillatory activity during transcranial magnetic stimulation , 2008, Human brain mapping.

[14]  Juha Silvanto,et al.  Visual feature binding: The critical time windows of V1/V2 and parietal activity , 2012, NeuroImage.

[15]  Kenzo Sakurai,et al.  Auditory Induced Bounce Perception Persists as the Probability of a Motion Reversal is Reduced , 2009, Perception.

[16]  D. H. Warren,et al.  Immediate perceptual response to intersensory discrepancy. , 1980, Psychological bulletin.

[17]  Alan Cowey,et al.  Temporal aspects of visual search studied by transcranial magnetic stimulation , 1997, Neuropsychologia.

[18]  Valerie Treyer,et al.  Time-course of “off-line” prefrontal rTMS effects — a PET study , 2008, NeuroImage.

[19]  Juha Silvanto,et al.  Relationship between visual binding, reentry and awareness , 2011, Consciousness and Cognition.

[20]  Neil G. Muggleton,et al.  The role of the angular gyrus in visual conjunction search investigated using signal detection analysis and transcranial magnetic stimulation , 2008, Neuropsychologia.

[21]  Shinsuke Shimojo,et al.  Crossmodal attention in event perception , 2005 .

[22]  C. Gerloff,et al.  Inhibitory control of acquired motor programmes in the human brain. , 2002, Brain : a journal of neurology.

[23]  J. Rothwell,et al.  Decreased corticospinal excitability after subthreshold 1 Hz rTMS over lateral premotor cortex , 2001, Neurology.

[24]  R. Sekuler,et al.  Sound alters visual motion perception , 1997, Nature.

[25]  Á. Pascual-Leone,et al.  Enhanced visual spatial attention ipsilateral to rTMS-induced 'virtual lesions' of human parietal cortex , 2001, Nature Neuroscience.

[26]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[27]  Marcello Massimini,et al.  A [17F]-fluoromethane PET/TMS study of effective connectivity , 2004, Brain Research Bulletin.

[28]  Alison R. Lane,et al.  The interaction of brain regions during visual search processing as revealed by transcranial magnetic stimulation. , 2007, Cerebral cortex.

[29]  F. Boiten,et al.  Event-related desynchronization: the effects of energetic and computational demands. , 1992, Electroencephalography and clinical neurophysiology.

[30]  Christopher Kennard,et al.  Differential cortical activation during voluntary and reflexive saccades in man , 2003, NeuroImage.

[31]  M. Hallett,et al.  Depression of motor cortex excitability by low‐frequency transcranial magnetic stimulation , 1997, Neurology.

[32]  M. Hallett,et al.  Neural correlates of cross-modal binding , 2003, Nature Neuroscience.

[33]  Peter T. Fox,et al.  Imaging human intra‐cerebral connectivity by PET during TMS , 1997, Neuroreport.

[34]  Vincent Walsh,et al.  The perceptual and functional consequences of parietal top-down modulation on the visual cortex. , 2009, Cerebral cortex.

[35]  B. Argall,et al.  Integration of Auditory and Visual Information about Objects in Superior Temporal Sulcus , 2004, Neuron.

[36]  Sven Bestmann,et al.  Concurrent TMS–fMRI reveals dynamic interhemispheric influences of the right parietal cortex during exogenously cued visuospatial attention , 2011, The European journal of neuroscience.

[37]  S Shimojo,et al.  Attentional Modulation in Perception of Visual Motion Events , 1998, Perception.

[38]  Yoshitaka Nakajima,et al.  Audiovisual integration: an investigation of the "streaming-bouncing" phenomenon. , 2004, Journal of physiological anthropology and applied human science.

[39]  John J. Foxe,et al.  Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. , 2004, Cerebral cortex.

[40]  C. Caltagirone,et al.  Modulation of excitatory and inhibitory circuits for visual awareness in the human right parietal cortex , 2004, Experimental Brain Research.

[41]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[42]  Takahiro Kawabe,et al.  Effects of the orientation of moving objects on the perception of streaming/bouncing motion displays , 2006, Perception & psychophysics.

[43]  Yousuke Kawachi,et al.  Presentation of a Visual Nearby Moving Object Alters Stream/Bounce Event Perception , 2006, Perception.

[44]  Tony Ro,et al.  Unconscious processing of orientation and color without primary visual cortex. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Renaud Brochard,et al.  Visual motion disambiguation by a subliminal sound , 2008, Consciousness and Cognition.

[46]  W. Metzger.  Beobachtungen über phänomenale Identität , 1934 .

[47]  Igor Schindler,et al.  Citation for Published Item: Use Policy an Exploration of the Role of the Superior Temporal Gyrus in Visual Search and Spatial Perception Using Tms. Running Title: the Role of Stg Lesions in Neglect-like Tasks , 2022 .

[48]  Bruno Weber,et al.  Lateralized and frequency-dependent effects of prefrontal rTMS on regional cerebral blood flow , 2006, NeuroImage.

[49]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[50]  Clara Casco,et al.  Audiovisual bounce-inducing effect: attention alone does not explain why the discs are bouncing. , 2009, Journal of experimental psychology. Human perception and performance.

[51]  R. Weale.  Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[52]  S. Campanella,et al.  Cross-modal interactions between human faces and voices involved in person recognition , 2011, Cortex.

[53]  G. W. Milligan,et al.  The Use of the Arc-Sine Transformation in the Analysis of Variance , 1987 .

[54]  A. Treisman,et al.  Parietal contributions to visual feature binding: evidence from a patient with bilateral lesions , 1995, Science.

[55]  P. Hazemann,et al.  Handbook of Electroencephalography and Clinical Neurophysiology , 1975 .

[56]  Robert Sekuler,et al.  Multi-sensory integration of spatio-temporal segmentation cues: one plus one does not always equal two , 2007, Experimental Brain Research.

[57]  Antonio Oliviero,et al.  The effects of subthreshold 1 Hz repetitive TMS on cortico-cortical and interhemispheric coherence , 2002, Clinical Neurophysiology.

[58]  S. Shimojo,et al.  When Sound Affects Vision: Effects of Auditory Grouping on Visual Motion Perception , 2001, Psychological science.

[59]  Clara Casco,et al.  Revealing the origin of the audiovisual bounce-inducing effect. , 2012, Seeing and perceiving.

[60]  Nikolaus Weiskopf,et al.  Hemispheric Differences in Frontal and Parietal Influences on Human Occipital Cortex: Direct Confirmation with Concurrent TMS–fMRI , 2009, Journal of Cognitive Neuroscience.

[61]  J. Schoenen,et al.  Effects of repetitive transcranial magnetic stimulation on visual evoked potentials: new insights in healthy subjects , 2003, Experimental Brain Research.