Interaction patterns for auditory user interfaces

This paper proposes the use of interaction patterns in the design process of auditory displays in human-computer interaction. To avoid introducing visual concepts in auditory design, a common ground for developing user interfaces without determining their means of representation is proposed. This meta domain allows for the design of user interfaces which can be equally realised in different interaction modalities or multi-modal settings. Although this work focuses on the auditory domain the concept shown is developed keeping in mind that it should be equally applicable in other modalities. A set of mode independent interaction patterns for design in the meta domain are introduced along with their transformation into the auditory domain. A real world application was chosen to evaluate the approach. MS Explorer was analysed, described through the mode independent interaction patterns and transformed into the auditory domain making extensive use of 3D audio rendering techniques. The result, a virtual audio reality version of a file manager, was evaluated with normally sighted persons as well as visually impaired and blind participants showing the feasibility and usability of the approach.

[1]  Hallvard Trætteberg,et al.  Interaction Patterns in User Interfaces. , 2000 .

[2]  Chris Schmandt,et al.  Audio hallway: a virtual acoustic environment for browsing , 1998, UIST '98.

[3]  Robert Höldrich,et al.  A 3D Ambisonic Based Binaural Sound Reproduction System , 2003 .

[4]  Robert Höldrich,et al.  A Spatial Audio Interface for Desktop Applications , 2003 .

[5]  Elizabeth D. Mynatt Transforming graphical interfaces into auditory interfaces , 1995, CHI 95 Conference Companion.

[6]  E. Verheijen Sound reproduction by wave field synthesis , 1998 .

[7]  Martin Kaltenbrunner Y-Windows: Proposal for a Standard AUI Environment , 2002 .

[8]  Christopher Frauenberger,et al.  SPATIAL AUDITORY DISPLAYS A STUDY ON THE USE OF VIRTUAL AUDIO ENVIRONMENTS AS INTERFACES FOR USERS WITH VISUAL DISABILITIES , 2004 .

[9]  David McGookin,et al.  Diary in the Sky: A Spatial Audio Display for a Mobile Calendar , 2001, BCS HCI/IHM.

[10]  W. Keith Edwards,et al.  The Mercator project: a nonvisual interface to the X Window system , 1993 .

[11]  Tapio Lokki,et al.  Creating Interactive Virtual Auditory Environments , 2002, IEEE Computer Graphics and Applications.

[12]  Christopher Frauenberger,et al.  A Generic, Semantically-Based Design Approach for Spatial Auditory Computer Displays , 2004, ICAD.

[13]  Robert Höldrich,et al.  3D binaural sound reproduction using a virtual ambisonic approach , 2003, IEEE International Symposium on Virtual Environments, Human-Computer Interfaces and Measurement Systems, 2003. VECIMS '03. 2003.

[14]  Jakob Nielsen,et al.  Usability engineering , 1997, The Computer Science and Engineering Handbook.

[15]  Ville Pulkki,et al.  Virtual Sound Source Positioning Using Vector Base Amplitude Panning , 1997 .

[16]  Meera Blattner,et al.  Earcons and Icons: Their Structure and Common Design Principles , 1989, Hum. Comput. Interact..

[17]  Keith Nesbitt,et al.  Modelling the Multi-Sensory Design Space , 2001, InVis.au.

[18]  Chris Schmandt,et al.  Dynamic Soundscape: mapping time to space for audio browsing , 1997, CHI.

[19]  Christopher Frauenberger,et al.  Mode independent interaction pattern design , 2005, Ninth International Conference on Information Visualisation (IV'05).

[20]  John G. Neuhoff,et al.  Sonification Report: Status of the Field and Research Agenda Prepared for the National Science Foundation by members of the International Community for Auditory Display , 1999 .

[21]  Gregory Kramer,et al.  Auditory Display: Sonification, Audification, And Auditory Interfaces , 1994 .

[22]  R A Lutfi,et al.  Informational masking by everyday sounds. , 1999, The Journal of the Acoustical Society of America.

[23]  Tapio Lokki,et al.  Comparison of auditory, visual, and audiovisual navigation in a 3D space , 2005, TAP.

[24]  David McGookin,et al.  AN INVESTIGATION INTO THE IDENTIFICATION OF CONCURRENTLY PRESENTED EARCONS , 2003 .

[25]  Stephen Barrass,et al.  SONIFICATION DESIGN PATTERNS , 2003 .

[26]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[27]  Matt Adcock,et al.  Cultivating Design Patterns for Auditory Displays , 2004, ICAD.