Notes on the Music: A social data infrastructure for music annotation

Beside transmitting musical meaning from composer to reader, symbolic music notation affords the dynamic addition of layers of information by annotation. This allows music scores to serve as rudimentary communication frameworks. Music encodings bring these affordances into the digital realm; though annotations may be represented as digital pen-strokes upon a score image, they must be captured using machine-interpretable semantics to fully benefit from this transformation. This is challenging, as annotators’ requirements are heterogeneous, varying both across different types of user (e.g., musician, scholar) and within these groups, depending on the specific use-case. A hypothetical all-encompassing tool catering to every conceivable annotation type, even if it were possible to build, would vastly complicate user interaction. This additional complexity would significantly increase cognitive load and impair usability, particularly in dynamic real-time usage contexts, e.g., live annotation during music rehearsal or performance. To address this challenge, we present a social data infrastructure that facilitates the creation of use-case specific annotation toolkits. Its components include a selectable-score module that supports customisable click-and-drag selection of score elements (e.g., notes, measures, directives); the Web Annotations data model, extended to support the creation of custom, Web-addressable annotation types supporting the specification and (re-)use of annotation palettes; and the Music Encoding and Linked Data (MELD) Javascript client library, used to build interfaces that map annotation types to rendering and interaction handlers. We have extended MELD to support the Solid platform for social Linked Data, allowing annotations to be privately stored in user-controlled Personal Online Datastores (Pods), or selectively shared or published. To demonstrate the feasibility of our proposed approach, we present annotation interfaces employing the outlined infrastructure in three distinct use-cases: scholarly communication; music rehearsal; and rating during music listening.

[1]  David De Roure,et al.  A Toolkit for Live Annotation of Opera Performance: Experiences Capturing Wagner's Ring Cycle , 2015, ISMIR.

[2]  Tim Berners-Lee,et al.  A Demonstration of the Solid Platform for Social Web Applications , 2016, WWW.

[3]  Kevin R. Page,et al.  Publishing musicology using multimedia digital libraries: creating interactive articles through a framework for linked data and MEI , 2018, DLfm.

[4]  Patricia Santos,et al.  Interweaving and Enriching Digital Music Collections for Scholarship, Performance, and Enjoyment , 2019, DLfM.

[5]  Jack Stilgoe,et al.  Citizen Scientists: Reconnecting Science with Civil Society , 2009 .

[6]  Kevin R. Page,et al.  Narratives and exploration in a musicology app: Supporting scholarly argument with the Lohengrin TimeMachine , 2021, DLfM.

[7]  Perry Roland,et al.  Verovio: A library for Engraving MEI Music Notation into SVG , 2014, ISMIR.

[8]  J. Russell A circumplex model of affect. , 1980 .

[9]  T. Eerola,et al.  A comparison of the discrete and dimensional models of emotion in music , 2011 .

[10]  C.C.S. Liem,et al.  Microtask crowdsourcing for music score Transcriptions: an experiment with error detection , 2020 .

[11]  Laurent Pugin Interaction Perspectives for Music Notation Applications , 2018, SAAM@ISWC.

[12]  Jeffrey J. Scott,et al.  MUSIC EMOTION RECOGNITION: A STATE OF THE ART REVIEW , 2010 .

[13]  Alex Hofmann,et al.  Read/Write Digital Libraries for Musicology , 2020, DLfM.

[14]  Kevin R. Page,et al.  Musicological Observations During Rehearsal and Performance: a Linked Data Digital Library for Annotations , 2019, DLfM.

[15]  Erik Schultes,et al.  The FAIR Guiding Principles for scientific data management and stewardship , 2016, Scientific Data.

[16]  Eita Nakamura,et al.  Performance Error Detection and Post-Processing for Fast and Accurate Symbolic Music Alignment , 2017, ISMIR.

[17]  The Cultural Significance of Timbre Analysis , 2020 .

[18]  Frans Wiering,et al.  In Their Own Words: Using Text Analysis to Identify Musicologists' Attitudes towards Technology , 2015, ISMIR.

[19]  Megan A. Winget,et al.  Annotations on musical scores by performing musicians: Collaborative models, interactive methods, and music digital library tool development , 2008, J. Assoc. Inf. Sci. Technol..

[20]  J. Silvertown A new dawn for citizen science. , 2009, Trends in ecology & evolution.

[21]  Elizabeth Grumbach,et al.  Music Scholarship Online (MuSO): A Research Environment for a More Democratic Digital Musicology , 2019, Digit. Humanit. Q..

[22]  Nicholas Cook The Ghost in the Machine: Towards a Musicology of Recordings , 2010 .

[23]  Mark B. Sandler,et al.  Do You sound like your Friends? Exploring Artist Similarity via Artist Social Network Relationships and audio signal Processing , 2009, ICMC.

[24]  Johanna Devaney,et al.  Using Note-Level Music Encodings to Facilitate Interdisciplinary Research on Human Engagement with Music , 2020, Trans. Int. Soc. Music. Inf. Retr..

[25]  Christophe Rhodes,et al.  Listen To Me - Don't Listen To Me: What Communities of Critics Tell Us About Music , 2016, ISMIR.

[26]  Meinard Müller,et al.  Towards Automatically Correcting Tapped Beat Annotations for Music Recordings , 2019, ISMIR.