Multisensory warning signals: when spatial correspondence matters

We report a study designed to investigate the effectiveness of task-irrelevant unimodal and bimodal audiotactile stimuli in capturing a person’s spatial attention away from a highly perceptually demanding central rapid serial visual presentation (RSVP) task. In “Experiment 1”, participants made speeded elevation discrimination responses to peripheral visual targets following the presentation of auditory stimuli that were either presented alone or else were paired with centrally presented tactile stimuli. The results showed that the unimodal auditory stimuli only captured spatial attention when participants were not performing the RSVP task, while the bimodal audiotactile stimuli did not result in any performance change in any of the conditions. In “Experiment 2”, spatial auditory stimuli were either presented alone or else were paired with a tactile stimulus presented from the same direction. In contrast to the results of “Experiment 1”, the bimodal audiotactile stimuli were especially effective in capturing participants’ spatial attention from the concurrent RSVP task. These results therefore provide support for the claim that auditory and tactile stimuli should be presented from the same direction if they are to capture attention effectively. Implications for multisensory warning signal design are discussed.

[1]  Alan H S Chan,et al.  Synchronous and asynchronous presentations of auditory and visual signals: Implications for control console design. , 2006, Applied ergonomics.

[2]  Ellen Poliakoff,et al.  The effect of visual threat on spatial attention to touch , 2007, Cognition.

[3]  Carter C. Collins,et al.  Tactile Television - Mechanical and Electrical Image Projection , 1970 .

[4]  Cristy Ho,et al.  Multisensory In-Car Warning Signals for Collision Avoidance , 2007, Hum. Factors.

[5]  Thomas K. Ferris,et al.  The Implications of Crossmodal Links in Attention for the Design of Multimodal Interfaces: A Driving Simulation Study , 2006 .

[6]  L. M. Ward,et al.  Cross-modal control of attention shifts , 1998 .

[7]  N. Lavie.  Distracted and confused?: Selective attention under load , 2005, Trends in Cognitive Sciences.

[8]  Jon Driver,et al.  A new approach to the design of multimodal warning signals , 1999 .

[9]  Lihua He,et al.  Functional degradation of visual cortical cells in old cats , 2006, Neurobiology of Aging.

[10]  C. Spence,et al.  Audiotactile temporal order judgments. , 2005, Acta psychologica.

[11]  M. Wallace,et al.  Enhanced multisensory integration in older adults , 2006, Neurobiology of Aging.

[12]  Charles Spence,et al.  Audiotactile multisensory interactions in human information processing , 2006 .

[13]  T. Stanford,et al.  Multisensory integration: current issues from the perspective of the single neuron , 2008, Nature Reviews Neuroscience.

[14]  A. Postma,et al.  Spatial attention triggered by unimodal, crossmodal, and bimodal exogenous cues: a comparison of reflexive orienting mechanisms , 2006, Experimental Brain Research.

[15]  B. Stein,et al.  The Merging of the Senses , 1993 .

[16]  C. Spence,et al.  The Handbook of Multisensory Processing , 2004 .

[17]  C. Spence,et al.  Exogenous spatial cuing studies of human crossmodal attention and multisensory integration , 2004 .

[18]  Mark T. Wallace,et al.  Crossmodal spatial interactions in subcortical and cortical circuits , 2004 .

[19]  C. Spence,et al.  Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision , 1998, Perception & psychophysics.

[20]  Paul J. Laurienti,et al.  Semantic congruence is a critical factor in multisensory behavioral performance , 2004, Experimental Brain Research.

[21]  Herbert J. Oyer,et al.  RESPONSE OF POPULATION TO OPTIMUM WARNING SIGNAL. , 1963 .

[22]  Robert W. Proctor,et al.  Effects of Displacement Magnitude and Direction of Auditory Cues on Auditory Spatial Facilitation of Visual Search , 2006, Hum. Factors.

[23]  C. Spence,et al.  Assessing the effectiveness of various auditory cues in capturing a driver's visual attention. , 2005, Journal of experimental psychology. Applied.

[24]  J. V. Erp,et al.  Vibrotactile in-vehicle navigation system , 2004 .

[25]  C. Spence,et al.  Tactile “capture” of audition , 2002, Perception & psychophysics.

[26]  Allen Allport,et al.  Visual attention , 1989 .

[27]  Charles Spence,et al.  Capturing spatial attention with multisensory cues , 2008, Psychonomic bulletin & review.

[28]  Charles Spence,et al.  Assessing the Automaticity of the Exogenous Orienting of Tactile Attention , 2007, Perception.

[29]  Charles Spence,et al.  Audiotactile interactions in near and far space , 2005, Experimental Brain Research.

[30]  Charles Spence,et al.  Head orientation biases tactile localization , 2007, Brain Research.

[31]  C. Spence,et al.  Crossmodal Space and Crossmodal Attention , 2004 .

[32]  C. Spence,et al.  Multisensory warning signals for event perception and safe driving , 2008 .

[33]  Riitta Hari,et al.  Hands help hearing: facilitatory audiotactile interaction at low sound-intensity levels. , 2004, The Journal of the Acoustical Society of America.

[34]  Frank P. McKenna,et al.  Integrating multiple information sources: using redundancy in the design of warnings , 1995 .

[35]  P Bertelson,et al.  Directing spatial attention towards the illusory location of a ventriloquized sound. , 2001, Acta psychologica.

[36]  Jacob Cohen.  Measurement Educational and Psychological Educational and Psychological Measurement Eta-squared and Partial Eta-squared in Fixed Factor Anova Designs Educational and Psychological Measurement Additional Services and Information For , 2022 .

[37]  Micah M. Murray,et al.  Auditory–somatosensory multisensory interactions in front and rear space , 2007, Neuropsychologia.

[38]  C. Spence,et al.  The suppression of reflexive visual and auditory orienting when attention is otherwise engaged. , 2007, Journal of experimental psychology. Human perception and performance.

[39]  F. Rösler,et al.  Crossmodal and intermodal attention modulate event-related brain potentials to tactile and auditory stimuli , 2002, Experimental Brain Research.

[40]  Brian M. Kleiner,et al.  Toward Developing an Approach for Alerting Drivers to the Direction of a Crash Threat , 2007, Hum. Factors.

[41]  J R Hennessy.  Cutaneous Sensitivity Communications1 , 1966, Human factors.

[42]  Cristy Ho,et al.  Tactile and Multisensory Spatial Warning Signals for Drivers , 2008, IEEE Transactions on Haptics.

[43]  John J. Foxe,et al.  Grabbing your ear: rapid auditory-somatosensory multisensory interactions in low-level sensory cortices are not constrained by stimulus alignment. , 2005, Cerebral cortex.

[44]  Charles Spence,et al.  Multisensory cues capture spatial attention regardless of perceptual load. , 2007, Journal of experimental psychology. Human perception and performance.

[45]  C. Spence,et al.  Is the exogenous orienting of spatial attention truly automatic? Evidence from unimodal and multisensory studies , 2008, Consciousness and Cognition.

[46]  Charles Spence,et al.  Perceptual load affects exogenous spatial orienting while working memory load does not , 2007, Experimental Brain Research.

[47]  C. Schroeder,et al.  Somatosensory input to auditory association cortex in the macaque monkey. , 2001, Journal of neurophysiology.

[48]  C. Spence,et al.  Attracting attention to the illusory location of a sound: reflexive crossmodal orienting and ventriloquism , 2000, Neuroreport.

[49]  Ana Tajadura-Jiménez,et al.  Auditory–somatosensory multisensory interactions are spatially modulated by stimulated body surface and acoustic spectra , 2009, Neuropsychologia.

[50]  C Spence,et al.  Multisensory interface design for drivers: past, present and future , 2008, Ergonomics.