Comparison of congruence judgment and auditory localization tasks for assessing the spatial limits of visual capture

Vision typically has better spatial accuracy and precision than audition and as a result often captures auditory spatial perception when visual and auditory cues are presented together. One determinant of visual capture is the amount of spatial disparity between auditory and visual cues: when disparity is small, visual capture is likely to occur, and when disparity is large, visual capture is unlikely. Previous experiments have used two methods to probe how visual capture varies with spatial disparity. First, congruence judgment assesses perceived unity between cues by having subjects report whether or not auditory and visual targets came from the same location. Second, auditory localization assesses the graded influence of vision on auditory spatial perception by having subjects point to the remembered location of an auditory target presented with a visual target. Previous research has shown that when both tasks are performed concurrently they produce similar measures of visual capture, but this may not hold when tasks are performed independently. Here, subjects alternated between tasks independently across three sessions. A Bayesian inference model of visual capture was used to estimate perceptual parameters for each session, which were compared across tasks. Results demonstrated that the range of audiovisual disparities over which visual capture was likely to occur was narrower in auditory localization than in congruence judgment, which the model indicates was caused by subjects adjusting their prior expectation that targets originated from the same location in a task-dependent manner.

[1]  H. Akaike A new look at the statistical model identification , 1974 .

[2]  Robert Hooke,et al.  `` Direct Search'' Solution of Numerical and Statistical Problems , 1961, JACM.

[3]  A. John Van Opstal,et al.  Plasticity in human sound localization induced by compressed spatial vision , 2003, Nature Neuroscience.

[4]  M. Wallace,et al.  Visual Localization Ability Influences Cross-Modal Bias , 2003, Journal of Cognitive Neuroscience.

[5]  L. Shams,et al.  The Brain’s Tendency to Bind Audiovisual Signals Is Stable but Not General , 2016, Psychological science.

[6]  Xue-Xin Wei,et al.  A Bayesian observer model constrained by efficient coding can explain 'anti-Bayesian' percepts , 2015, Nature Neuroscience.

[7]  U. Noppeney,et al.  Cortical Hierarchies Perform Bayesian Causal Inference in Multisensory Perception , 2015, PLoS biology.

[8]  Wardiya Afshar Saber,et al.  Modeling pain in vitro using nociceptor neurons reprogrammed from fibroblasts , 2014, Nature Neuroscience.

[9]  Ladan Shams,et al.  Bayesian priors are encoded independently from likelihoods in human multisensory perception. , 2009, Journal of vision.

[10]  N. Nagelkerke,et al.  A note on a general definition of the coefficient of determination , 1991 .

[11]  I. Howard,et al.  Human Spatial Orientation , 1966 .

[12]  Christophe Godin,et al.  A Computational Framework for 3D Mechanical Modeling of Plant Morphogenesis with Cellular Resolution , 2015, PLoS Comput. Biol..

[13]  M. Wallace,et al.  Unifying multisensory signals across time and space , 2004, Experimental Brain Research.

[14]  W R Thurlow,et al.  Effects of degree of visual association and angle of displacement on the "ventriloquism" effect. , 1973, Perceptual and motor skills.

[15]  P. Bertelson,et al.  Cross-modal bias and perceptual fusion with auditory-visual spatial discordance , 1981, Perception & psychophysics.

[16]  Salvador Soto-Faraco,et al.  Conscious access to the unisensory components of a cross-modal illusion , 2007, Neuroreport.

[17]  Antígona Martínez,et al.  Effect of Attention on Early Cortical Processes Associated with the Sound-induced Extra Flash Illusion , 2010, Journal of Cognitive Neuroscience.

[18]  J. Lewald,et al.  Cross-modal perceptual integration of spatially and temporally disparate auditory and visual stimuli. , 2003, Brain research. Cognitive brain research.

[19]  M. Godfroy,et al.  Spatial Variations of Visual—Auditory Fusion Areas , 2003, Perception.

[20]  Ladan Shams,et al.  Biases in Visual, Auditory, and Audiovisual Perception of Space , 2015, PLoS Comput. Biol..

[21]  D. H. Warren,et al.  The role of visual-auditory “compellingness” in the ventriloquism effect: Implications for transitivity among the spatial senses , 1981, Perception & psychophysics.

[22]  Kazuyuki Aihara,et al.  Bayesian Inference Explains Perception of Unity and Ventriloquism Aftereffect: Identification of Common Sources of Audiovisual Stimuli , 2007, Neural Computation.

[23]  W R Thurlow,et al.  Certain Determinants of the “Ventriloquism Effect” , 1973, Perceptual and motor skills.

[24]  G. Recanzone,et al.  Comparison of relative and absolute sound localization ability in humans. , 1998, The Journal of the Acoustical Society of America.

[25]  Robert A Jacobs,et al.  Bayesian integration of visual and auditory signals for spatial localization. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[26]  P. Legendre MODEL II REGRESSION USER’S GUIDE, R EDITION , 2008 .

[27]  David R. Wozny,et al.  Computational Characterization of Visually Induced Auditory Spatial Adaptation , 2011, Front. Integr. Neurosci..

[28]  G. Recanzone,et al.  Temporal and spatial dependency of the ventriloquism effect , 2001, Neuroreport.

[29]  Barbara G Shinn-Cunningham,et al.  Nothing Is Irrelevant in a Noisy World: Sensory Illusions Reveal Obligatory within-and across-Modality Integration , 2012, The Journal of Neuroscience.

[30]  Konrad Paul Kording,et al.  Causal Inference in Multisensory Perception , 2007, PloS one.

[31]  D. Burr,et al.  The Ventriloquist Effect Results from Near-Optimal Bimodal Integration , 2004, Current Biology.

[32]  Marc M Van Wanrooij,et al.  Acquired prior knowledge modulates audiovisual integration , 2010, The European journal of neuroscience.

[33]  Ulrik R. Beierholm,et al.  Probability Matching as a Computational Strategy Used in Perception , 2010, PLoS Comput. Biol..

[34]  Ladan Shams,et al.  The effects of selective and divided attention on sensory precision and integration , 2016, Neuroscience Letters.

[35]  William E O'Neill,et al.  Auditory Spatial Perception Dynamically Realigns with Changing Eye Position , 2007, The Journal of Neuroscience.

[36]  Marina S Dobreva,et al.  Influence of aging on human sound localization. , 2011, Journal of neurophysiology.

[37]  W M Hartmann,et al.  On the source-identification method. , 1998, The Journal of the Acoustical Society of America.

[38]  C. Schroeder,et al.  Contextual control of audiovisual integration in low‐level sensory cortices , 2014, Human brain mapping.

[39]  W. O'Neill,et al.  Influence of age, spatial memory, and ocular fixation on localization of auditory, visual, and bimodal targets by human subjects , 2012, Experimental Brain Research.