Towards building an experiential music visualizer

There have been many attempts to represent music using a wide variety of different music visualisation schemes. In this paper, we propose a novel system architecture which combines Max/MSPtrade with Flashtrade that can be used to build real-time music visualisations rapidly. In addition, we have used this proposed architecture to develop a music visualisation scheme that uses individual notes from a MIDI keyboard or from a standard MIDI file and creates novel displays that reflect pitch, note duration, characteristics such as how hard a key is struck, and which instruments are playing at any one time. The proposed music visualization scheme is a first step towards developing a music visualization that alone can provide a sense of musical experience to the user.

[1]  L E Marks,et al.  On associations of light and sound: the mediation of brightness, pitch, and loudness. , 1974, The American journal of psychology.

[2]  Ondřej Kubelka xkubelka Interactive music visualization , 2000 .

[3]  Norman I. Badler,et al.  Movies from music: Visualizing musical compositions , 1979, SIGGRAPH '79.

[4]  S. DiPaola,et al.  Affective communication remapping in MusicFace System , 2004 .

[5]  Daniel Torres,et al.  Visualizing Emotion in Musical Performance Using a Virtual Character , 2005, Smart Graphics.

[6]  E. Chew Towards a mathematical model of tonality , 2000 .

[7]  Daniel Torres,et al.  Using Music to Interact with a Virtual Character , 2005, NIME.

[8]  Tara Matthews,et al.  Visualizing non-speech sounds for the deaf , 2005, Assets '05.

[9]  James A. S. Angus,et al.  A comparison between singing pitching strategies of 8 to 11 year olds and trained adult singers , 1997 .

[10]  Geoff Wyvill,et al.  Visualization of musical pitch , 2003, Proceedings Computer Graphics International 2003.

[11]  M. Cardle,et al.  Music-driven motion editing: local motion transformations guided by music analysis , 2002, Proceedings 20th Eurographics UK Conference.

[12]  Stefan Koelsch,et al.  Cognitive priming in sung and instrumental music: Activation of inferior frontal cortex , 2006, NeuroImage.

[13]  Jamie Ward,et al.  Sound-Colour Synaesthesia: to What Extent Does it Use Cross-Modal Mechanisms Common to us All? , 2006, Cortex.

[14]  Densil Cabrera,et al.  Seeing sound: real-time sound visualisation in visual feedback loops used for training musicians , 2005, Ninth International Conference on Information Visualisation (IV'05).

[15]  James A. Landay,et al.  Can you see what i hear?: the design and evaluation of a peripheral sound display for the deaf , 2003, CHI '03.

[16]  Jonathan Foote,et al.  Visualizing music and audio using self-similarity , 1999, MULTIMEDIA '99.

[17]  Barry Vercoe,et al.  Music-listening systems , 2000 .

[18]  Issei Fujishiro,et al.  Music learning through visualization , 2002, Second International Conference on Web Delivering of Music, 2002. WEDELMUSIC 2002. Proceedings..

[19]  E. Chew Modeling Tonality: Applications to Music Cognition , 2001 .