Pip and Pop

Dynamic and complex command and control situations often require the timely recognition of changes in the environment in order to detect potentially malicious actions. Change detection can be challenging within a continually evolving scene, and particularly under multitasking conditions whereby attention is necessarily divided between several subtasks. On-screen tools can assist with detection (e.g., providing a visual record of changes, ensuring that none are overlooked), however, in a high workload environment, this may result in information overload to the detriment of the primary task. One alternative is to exploit the auditory modality as a means to support visual change detection. In the current study, we use a naval air-warfare simulation, and introduce an auditory alarm to coincide with critical visual changes (in aircraft speed/direction) on the radar. We found that participants detected a greater percentage of visual changes and were significantly quicker to detect these changes when they were accompanied by an auditory alarm than when they were not. Furthermore, participants reported that mental demand was lower in the auditory alarm condition, and this was reflected in reduced classification omissions on the primary task. Results are discussed in relation to Wickens’ multiple resource theory of attention and indicate the potential for using the auditory modality to facilitate visual change detection.

[1]  Jason B. Mattingley,et al.  Directed Attention Eliminates ‘Change Deafness’ in Complex Auditory Scenes , 2005, Current Biology.

[2]  I. Rock,et al.  Inattentional blindness: Perception without attention. , 1998 .

[3]  Ronald A. Rensink,et al.  TO SEE OR NOT TO SEE: The Need for Attention to Perceive Changes in Scenes , 1997 .

[4]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[5]  Zhuanghua Shi,et al.  Non-spatial sounds regulate eye movements and enhance visual search. , 2012, Journal of vision.

[6]  Christopher D. Wickens,et al.  Multiple Resources and Mental Workload , 2008, Hum. Factors.

[7]  J Edworthy,et al.  Improving Auditory Warning Design: Relationship between Warning Sound Parameters and Perceived Urgency , 1991, Human factors.

[8]  Dylan M. Jones,et al.  Nonexplicit Change Detection in Complex Dynamic Settings , 2012, Hum. Factors.

[9]  Michael S Vitevitch,et al.  Change deafness: the inability to detect changes between two voices. , 2003, Journal of experimental psychology. Human perception and performance.

[10]  Sébastien Tremblay,et al.  Supporting Change Detection in Complex Dynamic Situations: Does the CHEX Serve its Purpose? , 2012 .

[11]  Sébastien Tremblay,et al.  Background Sound Impairs Interruption Recovery in Dynamic Task Situations: Procedural Conflict? , 2014 .

[12]  J. Vroomen,et al.  Sound enhances visual perception: cross-modal effects of auditory organization on vision. , 2000, Journal of experimental psychology. Human perception and performance.

[13]  Daniel T. Levin,et al.  Unseen and Unaware: Implications of Recent Research on Failures of Visual Awareness for Human-Computer Interface Design , 2004, Hum. Comput. Interact..

[14]  N. Diarrassouba,et al.  Research article , 2001 .

[15]  C. Spence,et al.  Auditory, tactile, and multisensory cues facilitate search for dynamic visual stimuli , 2010, Attention, perception & psychophysics.

[16]  Dylan M. Jones,et al.  Exploiting the Auditory Modality in Decision Support , 2011 .

[17]  Harvey S. Smallman,et al.  Chex (Change History Explicit): New HCI Concepts for Change Awareness , 2003 .

[18]  Jan Theeuwes,et al.  Pip and pop: nonspatial auditory signals improve spatial visual search. , 2008, Journal of experimental psychology. Human perception and performance.

[19]  R. R. Patterson,et al.  Guidelines for auditory warning systems on civil aircraft , 1982 .