Non-Speech Sound in Human-Computer Interaction: A Review and Design Guidelines

Recent developments in computer technology make it possible to use sound more extensively in the computer interface than has hitherto been possible. This article describes research on uses of computer sound and suggests how sound might be used effectively by instructional and interface designers. After a review of some general principles of interface design and of basic research into auditory perception, the article examines two uses of sound: Earcons, whose purpose is to inform the user about the state of the system itself, and sonitization, whose purpose is to represent data generated by computer programs as sound. Both symbolic and iconic sounds are examined. The role of sound in virtual environments is also discussed with particular attention to its use in aircraft. Design guidelines are listed, and conclusions about needed research are offered.

[1]  Thomas A. Furness.  The Super Cockpit and its Human Factors Challenges , 1986 .

[2]  Janaf.  Human Engineering Guide to Equipment Design , 1984 .

[3]  William W. Gaver.  The SonicFinder: An Interface That Uses Auditory Icons , 1989, Hum. Comput. Interact..

[4]  William W. Gaver,et al.  Sound Support for Collaboration , 1991, ECSCW.

[5]  Ira J. Hirsh,et al.  Auditory Perception of Temporal Order , 1959 .

[6]  William W. Gaver,et al.  AUDITORY ICONS IN LARGE-SCALE COLLABORATIVE ENVIRONMENTS , 1990, SGCH.

[7]  Elizabeth M. Wenzel,et al.  Localization with non-individualized virtual acoustic display cues , 1991, CHI.

[8]  D. Gareth Loy,et al.  Musicians make a standard: the MIDI phenomenon , 1985 .

[9]  Scott S. Fisher,et al.  A system for three-dimensional acoustic 'visualization' in a virtual environment workstation , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[10]  Ira J. Hirsh,et al.  Temporal Order and Auditory Perception , 1974 .

[11]  A.D.N. Edwards.  Extending communication for users with restricted channels , 1990 .

[12]  E. Owens,et al.  An Introduction to the Psychology of Hearing , 1997 .

[13]  Mari Riess Jones,et al.  AUDITORY PATTERNS: STUDIES IN THE PERCEPTION OF STRUCTURE , 1978 .

[14]  C. Watson,et al.  Perception of complex auditory patterns by humans. , 1989 .

[15]  W. Dowling,et al.  Contour, interval, and pitch recognition in memory for melodies. , 1971, The Journal of the Acoustical Society of America.

[16]  Irvin Rock,et al.  The description and analysis of object and event perception. , 1986 .

[17]  D. Norman.  The psychology of everyday things", Basic Books Inc , 1988 .

[18]  R. L. Small,et al.  A real-time approach to information management in a pilot's associate , 1991, IEEE/AIAA 10th Digital Avionics Systems Conference.

[19]  Robert C. Morrison,et al.  A Microcomputer-Based Laboratory Aid for Visually Impaired Students , 1983, IEEE Micro.

[20]  Georges G. Grinstein,et al.  Exvis: an exploratory visualization environment , 1989 .

[21]  Alistair D. N. Edwards,et al.  Adapting user interfaces for visually disabled users , 1987 .

[22]  F L Wightman,et al.  Localization using nonindividualized head-related transfer functions. , 1993, The Journal of the Acoustical Society of America.

[23]  Alistair D. N. Edwards,et al.  Modelling Blind Users' Interactions with an Auditory Computer Interface , 1989, Int. J. Man Mach. Stud..

[24]  Alistair D. N. Edwards,et al.  Soundtrack: An Auditory Interface for Blind Users (Abstract Only) , 1989, SGCH.

[25]  R. Daniel Bergeron,et al.  Stereophonic and surface sound generation for exploratory data analysis , 1990, CHI '90.

[26]  N Durlach.  Auditory Localization in Teleoperator and Virtual Environment Systems: Ideas, Issues, and Problems , 1991, Perception.

[27]  Ephraim P. Glinert,et al.  Visual tools and languages: directions for the '90s , 1991, Proceedings 1991 IEEE Workshop on Visual Languages.

[28]  Preston S. Hall,et al.  Helmet-mounted systems technology planning for the future , 1992, Defense, Security, and Sensing.

[29]  William Winn,et al.  The effect of the spatial arrangement of simple diagrams on the interpretation of english and nonsense sentences , 1993 .

[30]  Meredith Bricken,et al.  Virtual worlds: no interface to design , 1991 .

[31]  Alan B. Craig,et al.  Using sound to extract meaning from complex data , 1991, Electronic Imaging.

[32]  W. Barfield,et al.  The use of icons, earcons, and commands in the design of an online hierarchical menu , 1991 .

[33]  C. Krumhansl.  Music as Cognition. , 1987 .

[34]  Meera Blattner,et al.  Earcons and Icons: Their Structure and Common Design Principles , 1989, Hum. Comput. Interact..

[35]  Elizabeth M. Wenzel,et al.  Development of a three-dimensional auditory display system , 1988, SGCH.

[36]  O. Reiser,et al.  Principles Of Gestalt Psychology , 1936 .

[37]  Erwin A. Ulbrich,et al.  Controls and displays for Douglas Aircraft for the 1990s , 1992, Electronic Imaging.

[38]  William W. Gaver,et al.  Effective sounds in complex systems: the ARKOLA simulation , 1991, CHI.

[39]  Erwin A. Ulbrich,et al.  Controls and displays for Douglas Aircraft for the 1990s , 1992, [1992] Proceedings IEEE/AIAA 11th Digital Avionics Systems Conference.

[40]  B. Moore.  An Introduction to the Psychology of Hearing , 1977 .

[41]  J. Mezrich,et al.  Dynamic Representation of Multivariate Time Series Data , 1984 .

[42]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[43]  Diane H. Sonnenwald,et al.  InfoSound: an audio aid to program comprehension , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[44]  Elizabeth M. Wenzel,et al.  Localization in Virtual Acoustic Displays , 1992, Presence: Teleoperators & Virtual Environments.

[45]  Durand R. Begault,et al.  Headphone Localization of Speech Stimuli , 1991 .

[46]  Carla Scaletti,et al.  The Kyma/Platypus Computer Music Workstation , 1989 .

[47]  Ephraim P. Glinert,et al.  Metawidgets: towards a theory of multimodal interface design , 1992, [1992] Proceedings. The Sixteenth Annual International Computer Software and Applications Conference.

[48]  William W. Gaver.  Auditory Icons: Using Sound in Computer Interfaces , 1986, Hum. Comput. Interact..