Amplitude-modulated auditory stimuli influence selection of visual spatial frequencies.

A recent study by Guzman-Martinez, Ortega, Grabowecky, Mossbridge, and Suzuki (2012) showed that participants match the frequency of an amplitude-modulated auditory stimulus to visual spatial frequency with a linear relationship and suggested this crossmodal matching guided attention to specific spatial frequencies. Here, we replicated this matching relationship and used the visual search paradigm to investigate whether auditory signals guide attention to matched visual spatial frequencies. Participants were presented with a search display of Gabors, all with different spatial frequencies. When the auditory signal was informative, improved search efficiency occurred for some spatial frequencies. However, when uninformative, a matched auditory signal produced no effect on visual search performance whatsoever. Moreover, search benefits were also observed when the auditory signal was informative, but did not match the spatial frequency. Together, these findings suggest that an amplitude-modulated auditory signal can influence visual selection of a matched spatial frequency, but the effect is due to top-down knowledge rather than resulting from automatic attentional capture derived from low-level mapping.

[1]  D Regan,et al.  Spatial frequency discrimination in normal vision and in patients with multiple sclerosis. , 1982, Brain : a journal of neurology.

[2]  P. Bertelson,et al.  The ventriloquist effect does not depend on the direction of deliberate visual attention , 2000, Perception & psychophysics.

[3]  C. Olivers,et al.  Bleeping you out of the blink: Sound saves vision from oblivion , 2008, Brain Research.

[4]  N. Bolognini,et al.  “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs , 2004, Experimental Brain Research.

[5]  J. Theeuwes,et al.  Poke and pop: Tactile–visual synchrony increases visual saliency , 2009, Neuroscience Letters.

[6]  de Gelder.  Sound Enhances Visual Perception: Cross-Modal Effects of Auditory Organization on Vision , 2001 .

[7]  John J. Foxe,et al.  Dual mechanisms for the cross-sensory spread of attention: how much do learned associations matter? , 2010, Cerebral cortex.

[8]  M. Giard,et al.  Auditory-Visual Integration during Multimodal Object Recognition in Humans: A Behavioral and Electrophysiological Study , 1999, Journal of Cognitive Neuroscience.

[9]  M Di Luca,et al.  Recalibration of audiovisual simultaneity , 2010 .

[10]  P. Bertelson,et al.  Recalibration of temporal order perception by exposure to audio-visual asynchrony. , 2004, Brain research. Cognitive brain research.

[11]  D. Senkowski,et al.  The multifaceted interplay between attention and multisensory integration , 2010, Trends in Cognitive Sciences.

[12]  John J. Foxe,et al.  Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. , 2005, Cerebral cortex.

[13]  Robert A Jacobs,et al.  Bayesian integration of visual and auditory signals for spatial localization. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[14]  D. Alais,et al.  Efficient Visual Search from Synchronized Auditory Signals Requires Transient Audiovisual Events , 2010, PLoS ONE.

[15]  Jan Theeuwes,et al.  The size of an attentional window modulates attentional capture by color singletons , 2007, Psychonomic bulletin & review.

[16]  Tatjana Seizova-Cajic,et al.  Illusory motion reversals from unambiguous motion with visual, proprioceptive, and tactile stimuli , 2008, Vision Research.

[17]  Nancy Kanwisher,et al.  fMRI evidence for objects as the units of attentional selection , 1999, Nature.

[18]  R. Sekuler,et al.  Sound alters visual motion perception , 1997, Nature.

[19]  Pascal Mamassian,et al.  Synchronized Audio-Visual Transients Drive Efficient Visual Search for Motion-in-Depth , 2011, PloS one.

[20]  John J. Foxe,et al.  Multisensory visual-auditory object recognition in humans: a high-density electrical mapping study. , 2004, Cerebral cortex.

[21]  Hans-Jochen Heinze,et al.  Sound increases the saliency of visual events , 2008, Brain Research.

[22]  P. Mamassian,et al.  Multisensory processing in review: from physiology to behaviour. , 2010, Seeing and perceiving.

[23]  N. Bolognini,et al.  Enhancement of visual perception by crossmodal visuo-auditory interaction , 2002, Experimental Brain Research.

[24]  M. Woldorff,et al.  Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? , 2006, Cerebral cortex.

[25]  D. Alais,et al.  Attending to auditory signals slows visual alternations in binocular rivalry , 2010, Vision Research.

[26]  Jan Theeuwes,et al.  Audiovisual semantic interference and attention: evidence from the attentional blink paradigm. , 2010, Acta psychologica.

[27]  Jan Theeuwes,et al.  Early multisensory interactions affect the competition among multiple visual objects , 2011, NeuroImage.

[28]  L. Busse,et al.  The spread of attention across modalities and space in a multisensory object. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[29]  Lawrence G. McDade,et al.  Behavioral Indices of Multisensory Integration: Orientation to Visual Cues is Affected by Auditory Stimuli , 1989, Journal of Cognitive Neuroscience.

[30]  J. Theeuwes.  Top-down and bottom-up control of visual selection. , 2010, Acta psychologica.

[31]  G. Aschersleben,et al.  Temporal ventriloquism: crossmodal interaction on the time dimension. 1. Evidence from auditory-visual temporal order judgment. , 2003, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[32]  R. Campbell,et al.  Audiovisual Integration of Speech Falters under High Attention Demands , 2005, Current Biology.

[33]  Erik Van der Burg,et al.  Sound speeds vision through preparation, not integration. , 2013, Journal of experimental psychology. Human perception and performance.

[34]  Jan Theeuwes,et al.  The Attentional Window Modulates Capture by Audiovisual Events , 2012, PloS one.

[35]  Yukio Iwaya,et al.  Alternation of Sound Location Induces Visual Motion Perception of a Static Object , 2009, PloS one.

[36]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[37]  Salvador Soto-Faraco,et al.  Attention to touch weakens audiovisual speech integration , 2007, Experimental Brain Research.

[38]  C. Spence,et al.  Attention and the crossmodal construction of space , 1998, Trends in Cognitive Sciences.

[39]  Yukio Iwaya,et al.  Sounds can alter the perceived direction of a moving visual object. , 2012, Journal of vision.

[40]  Marcia Grabowecky,et al.  Interactive Coding of Visual Spatial Frequency and Auditory Amplitude-Modulation Rate , 2012, Current Biology.

[41]  Antígona Martínez,et al.  Effect of Attention on Early Cortical Processes Associated with the Sound-induced Extra Flash Illusion , 2010, Journal of Cognitive Neuroscience.

[42]  J Hirsch,et al.  Limits of spatial-frequency discrimination as evidence of neural interpolation. , 1982, Journal of the Optical Society of America.

[43]  J. Theeuwes.  Perceptual selectivity for color and form , 1992, Perception & psychophysics.

[44]  John J. Foxe,et al.  Multisensory auditory-visual interactions during early sensory processing in humans: a high-density electrical mapping study. , 2002, Brain research. Cognitive brain research.

[45]  Roberto Arrighi,et al.  Meaningful auditory information enhances perception of visual biological motion. , 2009, Journal of vision.

[46]  C. Spence,et al.  Auditory, tactile, and multisensory cues facilitate search for dynamic visual stimuli , 2010, Attention, perception & psychophysics.

[47]  D. Burr,et al.  The Ventriloquist Effect Results from Near-Optimal Bimodal Integration , 2004, Current Biology.

[48]  John J. Foxe,et al.  Object‐based attention is multisensory: co‐activation of an object's representations in ignored sensory modalities , 2007, The European journal of neuroscience.

[49]  Jan Theeuwes,et al.  Pip and pop: nonspatial auditory signals improve spatial visual search. , 2008, Journal of experimental psychology. Human perception and performance.

[50]  David Alais,et al.  Multisensory Congruency as a Mechanism for Attentional Control over Perceptual Selection , 2009, The Journal of Neuroscience.