Utilizing Sound Effects in Mobile User Interface Design

The current generation of mobile devices is capable of producing polyphonic sounds, has enough processing power for real-time signal processing, and much better sound quality than their predecessors. The importance of audio is increasing as we are moving towards multimodal user interfaces where audio is one of the major components. In this paper, we present new ways of using audio feedback more efficiently and intelligently in mobile user interfaces by utilizing real-time signal processing. To test the ideas in practice, a prototype calendar application was implemented. We arranged a one week field trial to validate the design ideas. The results indicate that sound effects are capable of passing information to the user in some extent, but they are more useful in impressing the user and making existing audio feedback sound better.

[1]  Sara Bly,et al.  Presenting information in sound , 1982, CHI '82.

[2]  J. Mezrich,et al.  Dynamic Representation of Multivariate Time Series Data , 1984 .

[3]  Stephen Brewster,et al.  Designing non-speech sounds to support navigation in mobile phone menus , 2000 .

[4]  Stephen A. Brewster,et al.  Earcons as a Method of Providing Navigational Cues in a Menu Hierarchy , 1996, BCS HCI.

[5]  Grégory Leplâtre,et al.  Menu sonification in a mobile phone – A prototype study. , 2001 .

[6]  William W. Gaver,et al.  Effective sounds in complex systems: the ARKOLA simulation , 1991, CHI.

[7]  Catherine M. Burns,et al.  Do You Hear What I Hear?: Reflecting on Auditory Display in Medicine , 2006 .

[8]  William W. Gaver,et al.  Chapter 42 – Auditory Interfaces , 1997 .

[9]  Ephraim P. Glinert,et al.  An experiment into the use of auditory cues to reduce visual workload , 1989, CHI '89.

[10]  Schloss Birlinghoven,et al.  Auditory seismology on free oscillations, focal mechanisms, explosions and synthetic seismograms , 2002 .

[11]  Mark Kahrs,et al.  Applications of digital signal processing to audio and acoustics , 1998 .

[12]  Helge Ritter,et al.  BROADCASTING AUDITORY WEATHER REPORTS - A PILOT PROJECT , 2003 .

[13]  John G. Neuhoff,et al.  Sonification Report: Status of the Field and Research Agenda Prepared for the National Science Foundation by members of the International Community for Auditory Display , 1999 .

[14]  Petr Janata,et al.  Marketbuzz: Sonification of Real-Time Financial Data , 2004, ICAD.

[15]  Craig T. Jin,et al.  Reverberation for Ambient Data Communication , 2004, ICAD.

[16]  Jean Laroche.  Time and Pitch Scale Modification of Audio Signals , 2002 .

[17]  Petr Janata,et al.  Marketbuzz: Sonification of real-time financial dataa , 2004 .

[18]  Jukka Holm.  Pitch-shifting methods for wavetable synthesis , 2000 .

[19]  Stephen A. Brewster,et al.  An evaluation of earcons for use in auditory human-computer interfaces , 1993, INTERCHI.