Educational Sound Symbols for the Visually Impaired

Acoustic-based computer interactivity offers great potential [1], particularly with blind and visually impaired users [2]. At Indiana University's School of Informatics at IUPUI, we have developed an innovative educational approach relying on "audemes," short, nonverbal sound symbols made up of 2-5 individual sounds lasting 3-7 seconds - like expanded "earcons"[3] - to encode and prompt memory. To illustrate: An audeme for "American Civil War" includes a 3-second snippet of the song Dixie partially overlapped by a snippet of Battle Hymn of the Republic , followed by battle sounds, together lasting 5 seconds. Our focus on non-verbal sound explores the mnemonic impact of metaphoric rather than literal signification. Working for a year with BVI students, we found audemes improved encoding and long-term memory of verbal educational content, even after five months, and engaged the students in stimulating ways.

[1]  Elizabeth D. Mynatt.  Designing with auditory icons: how well do we identify auditory cues? , 1994, CHI Conference Companion.

[2]  Jaime Sánchez,et al.  Interactive virtual acoustic environments for blind children: computing, usability, and cognition , 2001, CHI Extended Abstracts.

[3]  Alistair D. N. Edwards,et al.  Modelling Blind Users' Interactions with an Auditory Computer Interface , 1989, Int. J. Man Mach. Stud..

[4]  Jaime Sánchez,et al.  AudioMath: Blind children learning mathematics through audio , 2005 .

[5]  Dylan M. Jones,et al.  Indispensable benefits and unavoidable costs of unattended sound for cognitive functioning. , 2003, Noise & health.

[6]  J. Stevens,et al.  The Origin of Consciousness in the Breakdown of the Bicameral Mind by , 1978, Neurology.

[7]  Jaime Sánchez,et al.  Memory enhancement through audio , 2003, ASSETS.

[8]  A D Baddeley,et al.  Short-term Memory for Word Sequences as a Function of Acoustic, Semantic and Formal Similarity , 1966, The Quarterly journal of experimental psychology.

[9]  Stéphane Conversy.  Ad-hoc synthesis of auditory icons , 1998 .

[10]  M. Lassonde,et al.  Blind subjects process auditory spectral cues more efficiently than sighted individuals , 2004, Experimental Brain Research.

[11]  Gary P. Scavone,et al.  The sonic mapper: An interactive program for obtaining similarity ratings with auditory stimuli , 2002 .

[12]  Maribeth Back,et al.  Micro-narratives in sound design: Context, character, and caricature in waveform manipulation , 1996 .

[13]  R. Mayer,et al.  A coherence effect in multimedia learning: The case for minimizing irrelevant sounds in the design of multimedia instructional messages. , 2000 .

[14]  Meera Blattner,et al.  Earcons and Icons: Their Structure and Common Design Principles , 1989, Hum. Comput. Interact..

[15]  Janan Al-Awar Smither.  Short term memory demands in processing synthetic speech by old and young adults , 1993, Behav. Inf. Technol..

[16]  Jaime Sánchez.  Interactive virtual environments for blind children : usability and cognition , 2003 .

[17]  Stephen Brewster,et al.  Providing a Structured Method for Integrating Non-Speech Audio into Human-Computer Interfaces , 1994 .

[18]  Mexhid Ferati,et al.  Using Audemes as a Learning Medium for the Visually Impaired , 2009, HEALTHINF.

[19]  Tony Stockman.  AN EXAMINATION OF MECHANISMS TO COMBINE SPEECH AND SOUND FOR DATA ANALYSIS Demo paper for the ICAD05 workshop "Combining Speech and Sound in the User Interface" , 2005 .

[20]  William W. Gaver.  The SonicFinder: An Interface That Uses Auditory Icons , 1989, Hum. Comput. Interact..

[21]  Douglas Turnbull,et al.  Modeling the Semantics of Sound , 2007 .

[22]  Stephen Brewster,et al.  Providing an audio glance at algebra for blind readers , 1994 .