Ad-Hoc Access to Musical Sound for Deaf Individuals

Learning a musical instrument can be a challenging task for a deaf person due to limited access to sound. Prior work has developed visual and vibrotactile approaches to provide music-to-sound feedback to deaf people. However, these systems are not designed for ad-hoc access to sound, which enables a deaf person to explore sound from various audio sources and receive real-time feedback. In this paper we present the development of a music sensory substitution system that enables ad-hoc access to musical sounds. It provides the technical basis to study deeper research questions about understanding and creating sound.