Can you see what i hear?: the design and evaluation of a peripheral sound display for the deaf

We developed two visual displays for providing awareness of environmental audio to deaf individuals. Based on fieldwork with deaf and hearing participants, we focused on supporting awareness of non-speech audio sounds such as ringing phones and knocking in a work environment. Unlike past work, our designs support both monitoring and notification of sounds, support discovery of new sounds, and do not require a priori knowledge of sounds to be detected. Our Spectrograph design shows pitch and amplitude, while our Positional Ripples design shows amplitude and location of sounds. A controlled experiment involving deaf participants found neither display to be significantly distracting. However, users preferred the Positional Ripples display and found that display easier to monitor (notification sounds were detected with 90% success in a laboratory setting). The Spectrograph display also supported successful detection in most cases, and was well received when deployed in the field.