CymaSense: A Real-Time 3D Cymatics-Based Sound Visualisation Tool

What does music look like? Representation of music has taken many forms over time, from musical notation [16], through to random algorithm-based visualisations based on the amplitude of an audio signal [19]. One aspect of music visualisation that has not been widely explored is that of Cymatics. Cymatics are physical impressions of music created in mediums such as water. Current Cymatic visualisations are restricted to 2D imaging, whilst 3D visualisations of music are generally based on arbitrary mapping of audio-visual attributes. This paper looks at the design of CymaSense, an interactive tool based on Cymatics.

[1]  Robert Kosara,et al.  Visualization Criticism - The Missing Link Between Information Visualization and Art , 2007, 2007 11th International Conference Information Visualization (IV '07).

[2]  Sean M. Smith,et al.  A visualization of music , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[3]  Richard Rastall,et al.  The Notation of Western Music: An Introduction , 1982 .

[4]  Scott D. Lipscomb,et al.  PERCEIVED MATCH BETWEEN VISUAL PARAMETERS AND AUDITORY CORRELATES: AN EXPERIMENTAL MULTIMEDIA INVESTIGATION , 2004 .

[5]  Kostas Giannakis,et al.  4 Imaging Soundscapes : Identifying Cognitive Associations between Auditory and Visual Dimensions , 2000 .

[6]  Kostas Giannakis,et al.  A comparative evaluation of auditory-visual mappings for sound visualisation , 2006, Organised Sound.

[7]  William W. Gaver.  Auditory Icons: Using Sound in Computer Interfaces , 1986, Hum. Comput. Interact..

[8]  Mats B. Küssner,et al.  Shape, drawing and gesture : cross-modal mappings of sound and music , 2014 .

[9]  Martin Hachet,et al.  Interacting with 3D Reactive Widgets for Musical Performance , 2011 .

[10]  Ronna S. Kaplan,et al.  An analysis of music therapy program goals and outcomes for clients with diagnoses on the autism spectrum. , 2005, Journal of music therapy.

[11]  Lilia B. Villafuerte,et al.  Acquisition of social abilities through musical tangible user interface: children with autism spectrum condition and the reactable , 2012, CHI Extended Abstracts.

[12]  Mitchell Whitelaw,et al.  Synesthesia and Cross-Modality in Contemporary Audiovisuals , 2008 .

[13]  Lizbeth Escobedo,et al.  SensoryPaint: a multimodal sensory intervention for children with neurodevelopmental disorders , 2014, UbiComp.

[14]  Meera Blattner,et al.  Earcons and Icons: Their Structure and Common Design Principles , 1989, Hum. Comput. Interact..