MoshViz: A Detail+Overview Approach to Visualize Music Elements

A music piece contains a large amount of information represented as a series of instructions corresponding to notes that must be played at specific times. These simple notes are combined to form complex harmonic structures that can be difficult to identify and analyze. Due to its simplicity and straightforward interpretation, music sheets and piano rolls have been the visual metaphor employed by most music visualization tools to support interpretation. Albeit it can represent all necessary elements to perform a music piece, these metaphors do not explicitly show many of the patterns and structures inherent to music arrangements, such as rhythm progression and harmonic interactions, needing users to create a mental model of them. Moreover, comparing different pieces and visualizing how a particular instrument track relates to the others is an issue not only for music sheet-based techniques, but also for most existing music visualization methods. In this paper, we present a novel visualization framework, called Music Overview, Stability, and Harmony Visualization (MoshViz), which facilitates the visualization and understanding of music renditions, focusing mainly on the visual analysis of specific musical instruments. Our approach creates a high-level model of music data and highlights structures of interest, enabling a detail+overview visualization to assist users in the task of identifying harmonic and melodic patterns. The usefulness and representativeness of MoshViz are confirmed by a set of user tests which demonstrate that the proposed visual metaphor matches, with a high degree of accuracy, the mental model of different users regarding the recognizable patterns of sounds.

[1]  Jock D. Mackinlay,et al.  Cone Trees: animated 3D visualizations of hierarchical information , 1991, CHI.

[2]  Franc Solina,et al.  Visualization of concurrent tones in music with colours , 2010, ACM Multimedia.

[3]  Issei Fujishiro,et al.  BRASS: Visualizing Scores for Assisting Music Learning , 2003, ICMC.

[4]  Issei Fujishiro,et al.  comp-i: A System for Visual Exploration and Editing of MIDI Datasets , 2004, ICMC.

[5]  Daniel A. Keim,et al.  Literature Fingerprinting: A New Method for Visual Literary Analysis , 2007, 2007 IEEE Symposium on Visual Analytics Science and Technology.

[6]  W. Berry.  Structural functions in music , 1976 .

[7]  Takayuki Itoh,et al.  Colorscore -- Visualization and Condensation of Structure of Classical Music , 2011, 2011 15th International Conference on Information Visualisation.

[8]  Sean M. Smith,et al.  A visualization of music , 1997, IEEE Visualization.

[9]  Marti A. Hearst,et al.  ImproViz: visual explorations of jazz improvisations , 2005, CHI EA '05.

[10]  Huamin Qu,et al.  Visualizing the Semantic Structure in Classical Music Works , 2010, IEEE Transactions on Visualization and Computer Graphics.

[11]  Karrie Karahalios,et al.  Isochords: visualizing structure in music , 2007, GI '07.

[12]  William Rothstein,et al.  Phrase Rhythm in Tonal Music , 1990 .

[13]  Martin Wattenberg,et al.  Arc diagrams: visualizing structure in strings , 2002, IEEE Symposium on Information Visualization, 2002. INFOVIS 2002..

[14]  Vlado Keselj,et al.  MIDIVIS: VISUALIZING MUSIC PIECES STRUCTURE VIA SIMILARITY MATRICES , 2009 .

[15]  Roger F. Malina,et al.  Music Animation Machine Demonstration Reel , 1991 .

[16]  Michael John Hewitt.  Music theory for computer musicians , 2008 .

[17]  Elaine Chew,et al.  Visualizing Music: Tonal Progressions and Distributions , 2007, ISMIR.