Changes in auditory frequency guide visual–spatial attention

How do the characteristics of sounds influence the allocation of visual-spatial attention? Natural sounds typically change in frequency. Here we demonstrate that the direction of frequency change guides visual-spatial attention more strongly than the average or ending frequency, and provide evidence suggesting that this cross-modal effect may be mediated by perceptual experience. We used a Go/No-Go color-matching task to avoid response compatibility confounds. Participants performed the task either with their heads upright or tilted by 90°, misaligning the head-centered and environmental axes. The first of two colored circles was presented at fixation and the second was presented in one of four surrounding positions in a cardinal or diagonal direction. Either an ascending or descending auditory-frequency sweep was presented coincident with the first circle. Participants were instructed to respond to the color match between the two circles and to ignore the uninformative sounds. Ascending frequency sweeps facilitated performance (response time and/or sensitivity) when the second circle was presented at the cardinal top position and descending sweeps facilitated performance when the second circle was presented at the cardinal bottom position; there were no effects of the average or ending frequency. The sweeps had no effects when circles were presented at diagonal locations, and head tilt entirely eliminated the effect. Thus, visual-spatial cueing by pitch change is narrowly tuned to vertical directions and dominates any effect of average or ending frequency. Because this cross-modal cueing is dependent on the alignment of head-centered and environmental axes, it may develop through associative learning during waking upright experience.

[1]  Robert D. Melara,et al.  Interaction between synesthetically corresponding dimensions. , 1987 .

[2]  M. Grabowecky,et al.  Motion induced pitch: a case of visual-auditory synesthesia , 2010 .

[3]  J. Pratt,et al.  Symbolic control of visual attention: The role of working memory and attentional control settings. , 2003, Journal of experimental psychology. Human perception and performance.

[4]  T. Papathomas,et al.  Audiovisual short-term influences and aftereffects in motion: examination across three sets of directional pairings. , 2008, Journal of vision.

[5]  Kristina J. Nielsen,et al.  Object features used by humans and monkeys to identify rotated shapes. , 2008, Journal of vision.

[6]  M. Tervaniemi,et al.  Binding symbols and sounds: evidence from event-related oscillatory gamma-band activity. , 2007, Cerebral cortex.

[7]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[8]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[9]  M. Grabowecky,et al.  Characteristic sounds make you look at target objects more quickly , 2010, Attention, perception & psychophysics.

[10]  I H Bernstein,et al.  Effects of some variations in auditory input upon visual choice reaction time. , 1971, Journal of experimental psychology.

[11]  Heinz Werner,et al.  Studies in Physiognomic Perception: V. Effect of Ascending and Descending Gliding Tones on Autokinetic Motion , 1958 .

[12]  Anne Treisman,et al.  Natural cross-modal mappings between visual and auditory features. , 2011, Journal of vision.

[13]  Ryota Kanai,et al.  Changing pitch induced visual motion illusion , 2004, Current Biology.

[14]  M. Grabowecky,et al.  Characteristic sounds facilitate visual search , 2008, Psychonomic bulletin & review.

[15]  Z. Eitan,et al.  HOW MUSIC MOVES: Musical Parameters and Listeners' Images of Motion , 2006 .

[16]  R Walker,et al.  The effects of culture, environment, age, and musical training on choices of visual metaphors for sound , 1987, Perception & psychophysics.

[17]  Edward A. Vessel,et al.  Beauty and the beholder: highly individual taste for abstract, but not real-world images. , 2010, Journal of vision.

[18]  Yuanye Ma,et al.  Aging affects the neural representation of speed in Macaque area MT. , 2009, Cerebral cortex.

[19]  L E Marks,et al.  On cross-modal similarity: auditory-visual interactions in speeded discrimination. , 1987, Journal of experimental psychology. Human perception and performance.

[20]  N. Bolognini,et al.  “Acoustical vision” of below threshold stimuli: interaction among spatially converging audiovisual inputs , 2004, Experimental Brain Research.

[21]  Philip T Quinlan,et al.  Garner and congruence effects in the speeded classification of bimodal signals. , 2002, Journal of experimental psychology. Human perception and performance.

[22]  C. Spence,et al.  Attention and the crossmodal construction of space , 1998, Trends in Cognitive Sciences.

[23]  Lawrence E. Marks,et al.  Visual-auditory interaction in speeded classification: Role of stimulus difference , 1995, Perception & psychophysics.

[24]  D. Senkowski,et al.  The multifaceted interplay between attention and multisensory integration , 2010, Trends in Cognitive Sciences.

[25]  Lawrence G. McDade,et al.  Behavioral Indices of Multisensory Integration: Orientation to Visual Cues is Affected by Auditory Stimuli , 1989, Journal of Cognitive Neuroscience.

[26]  N. Bolognini,et al.  Enhancement of visual perception by crossmodal visuo-auditory interaction , 2002, Experimental Brain Research.

[27]  D. Deutsch,et al.  The glissando illusion and handedness , 2007, Neuropsychologia.

[28]  M. Tervaniemi,et al.  From symbols to sounds: visual symbolic information activates sound representations. , 2004, Psychophysiology.

[29]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[30]  C. C. Pratt The spatial character of high and low tones. , 1930 .

[31]  Nicholas J. Priebe,et al.  The Neural Representation of Speed in Macaque Area MT/V5 , 2003, The Journal of Neuroscience.