Visualized sound effect icons for improved multimedia accessibility: A pilot study

Abstract Sound effects are often used to communicate important information in multimedia such as video games. For instance, they may tell the player that a character has just snuck up on them, is firing at them, or is about to paddle over a waterfall. Nevertheless, there are times when playing sound may be inappropriate, may be inaudible, may become fatiguing and/or may be inaccessible for hard of hearing and deaf users. Therefore, an alternative to sound that can relay the same information would be beneficial to many users. The majority of studies into alternative presentations of sound for these purposes have focused on dialogue at the expense of music and sound effects. The paper introduces a pilot study of “SoundSign”, a prototype symbolic representation of sound effects for multimedia, using an innovative icon and compass that indicates direction, sound cue and proximity. Users who have disabled the sound, are hearing-impaired or are otherwise unable to hear sound will still get the information needed. A description of SoundSign and the results of a usability test are presented.

[1]  Heinz Ulrich Hoppe,et al.  AudioBattleship: blind learners collaboration through sound , 2003, CHI Extended Abstracts.

[2]  Lennart E. Nacke,et al.  More than a feeling: Measurement of sonic user experience and psychophysiology in a first-person shooter game , 2010, Interact. Comput..

[3]  Lale Akarun,et al.  Gestural Interfaces for Hearing-Impaired Communication , 2008 .

[4]  Kristine Jørgensen,et al.  Left in the dark: playing computer games with the sound turned off , 2008 .

[5]  Sarah A. Douglas,et al.  Finger dance: a sound game for blind people , 2007, Assets '07.

[6]  Michel Chion,et al.  Audio-Vision: Sound on Screen , 1994 .

[7]  E. Casey Wardynski,et al.  Introducing Emotion into Military Simulation and Video Game Design America's Army: Operations and VIRTE. , 2002 .

[8]  Dan Gärdenfors,et al.  Audio games: new perspectives on game audio , 2004, ACE '04.

[9]  A. Blake.  Game Sound, An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design , 2009 .

[10]  Klaus Miesenberger,et al.  More Than Just a Game: Accessibility in Computer Games , 2008, USAB.

[12]  Deborah I. Fels,et al.  Methods for inclusion: Employing think aloud protocols in software usability studies with individuals who are deaf , 2006, Int. J. Hum. Comput. Stud..

[13]  Steve Caplin,et al.  Icon Design: Graphic Icons in Computer Interface Design , 2001 .

[14]  Kelsi Lenz,et al.  Examining the Critical User Interface Components of First-Person Shooter (FPS) Games , 2008 .

[15]  Deborah I. Fels,et al.  Emotive captioning , 2007, CIE.

[16]  André P. Calitz,et al.  The added value of eye tracking in the usability evaluation of a network management tool , 2005 .

[17]  Dimitrios Tzovaras.  Multimodal User Interfaces , 2008 .

[18]  Tara Matthews,et al.  Visualizing non-speech sounds for the deaf , 2005, Assets '05.

[19]  Klaus Miesenberger,et al.  A Computer Game Designed for All , 2008, ICCHP.

[20]  Nicolas Monmarché,et al.  MP3 Players and Audio Games: An Alternative to Portable Video Games Console for Visually Impaired Players , 2008, ICCHP.