Augmenting Music Sheets with Harmonic Fingerprints

Common Music Notation (CMN) is the well-established foundation for the written communication of musical information, such as rhythm or harmony. CMN suffers from the complexity of its visual encoding and the need for extensive training to acquire proficiency and legibility. While alternative notations using additional visual variables (e.g., color to improve pitch identification) have been proposed, the community does not readily accept notation systems that vary widely from the CMN. Therefore, to support student musicians in understanding harmonic relationships, instead of replacing the CMN, we present a visualization technique that augments digital sheet music with a harmonic fingerprint glyph. Our design exploits the circle of fifths, a fundamental concept in music theory, as visual metaphor. By attaching such glyphs to each bar of a composition we provide additional information about the salient harmonic features available in a musical piece. We conducted a user study to analyze the performance of experts and non-experts in an identification and comparison task of recurring patterns. The evaluation shows that the harmonic fingerprint supports these tasks without the need for close-reading, as when compared to a not-annotated music sheet.

[1]  Toby Burrows,et al.  The Etymologies of Isidore of Seville , 2008 .

[2]  Samuel. Hunt,et al.  How can music visualisation techniques reveal different perspectives on musical structure , 2017 .

[3]  Greta Franzini,et al.  On Close and Distant Reading in Digital Humanities: A Survey and Future Challenges , 2015, EuroVis.

[4]  Lee Brasseur,et al.  Florence Nightingale's Visual Rhetoric in the Rose Diagrams , 2005 .

[5]  Karrie Karahalios,et al.  Isochords: visualizing structure in music , 2007, GI '07.

[6]  Vlado Keselj,et al.  Midivis: Visualizing Music Structure via Similarity Matrices , 2009, ICMC.

[7]  Delfina Malandrino,et al.  Evaluation Study of Visualisations for Harmonic Analysis of 4-Part Music , 2018, 2018 22nd International Conference Information Visualisation (IV).

[8]  Craig Stuart Sapp Visual hierarchical key analysis , 2005, CIE.

[9]  Daniel A. Keim,et al.  Literature Fingerprinting: A New Method for Visual Literary Analysis , 2007, 2007 IEEE Symposium on Visual Analytics Science and Technology.

[10]  Sean M. Smith,et al.  A visualization of music , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[11]  Didier Sébastien,et al.  Annotating Works for Music Education: Propositions for a Musical Forms and Structures Ontology and a Musical Performance Ontology , 2013, ISMIR.

[12]  Eric Taylor,et al.  The AB Guide to Music Theory , 1989 .

[13]  Hope R Strayer,et al.  From Neumes to Notes: The Evolution of Music Notation , 2013 .

[14]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[15]  Christopher Ariza,et al.  Music21: A Toolkit for Computer-Aided Musicology and Symbolic Music Data , 2010, ISMIR.

[16]  Issei Fujishiro,et al.  Exploring MIDI datasets , 2003, SIGGRAPH '03.

[17]  Martin Wattenberg,et al.  Arc diagrams: visualizing structure in strings , 2002, IEEE Symposium on Information Visualization, 2002. INFOVIS 2002..

[18]  Ming-Chuen Chuang,et al.  A proposal of a color music notation system on a single melody for music beginners , 2013 .

[19]  Marti A. Hearst,et al.  ImproViz: visual explorations of jazz improvisations , 2005, CHI EA '05.

[20]  Delfina Malandrino,et al.  Visualization and Music Harmony: Design, Implementation, and Evaluation , 2018, 2018 22nd International Conference Information Visualisation (IV).

[21]  Delfina Malandrino,et al.  Understanding the structure of musical compositions: Is visualization an effective approach? , 2017, Inf. Vis..

[22]  Michael Good MusicXML: An internet-friendly format for sheet music , 2001 .

[23]  Franc Solina,et al.  Visualization of concurrent tones in music with colours , 2010, ACM Multimedia.

[24]  Remco C. Veltkamp,et al.  Automatic Functional Harmonic Analysis , 2013, Computer Music Journal.

[25]  Daniel A. Keim,et al.  Analyzing Visual Mappings of Traditional and Alternative Music Notation , 2018, ArXiv.

[26]  Gerrit C. van der Veer,et al.  CHI '05 Extended Abstracts on Human Factors in Computing Systems , 2005, CHI 2005.