Seen music: ambient music data visualization for children with hearing impairments

In this paper, we propose a prototype of music visualization system that captures and records the music component into digital data form, and then displays the data in visual form for children with hearing impairments. The analog sound data of music played physically is scaled into a binary matrix and scalar values that is then used as data structures for transcribing the output. We designed a system that detects tune and speed from a physical violin, and demonstrated three tangible music visualizations that children see in their daily lives, employing a flowerpot, plants and a picture of frame. We describe how the data captured from physical musical instruments can be seen through these objects, and suggest future possibilities for interactive sound visualization in music education for children with hearing impairments.