Retrieving audio recordings using musical themes

In 1948, Barlow and Morgenstern released a collection of about 10,000 themes of well-known instrumental pieces from the corpus of Western Classical music [1]. These monophonic themes (usually four bars long) are often the most memorable parts of a piece of music. In this paper, we report on a systematic study considering a cross-modal retrieval scenario. Using a musical theme as a query, the objective is to identify all related music recordings from a given audio collection. By adapting well-known retrieval techniques, our main goal is to get a better understanding of the various challenges including tempo deviations, musical tunings, key transpositions, and differences in the degree of polyphony between the symbolic query and the audio recordings to be retrieved. In particular, we present an oracle fusion approach that indicates upper performance limits achievable by a combination of current retrieval techniques.

[1]  Remco C. Veltkamp,et al.  A Survey of Music Information Retrieval Systems , 2005, ISMIR.

[2]  Marc Leman,et al.  Content-Based Music Information Retrieval: Current Directions and Future Challenges , 2008, Proceedings of the IEEE.

[3]  Matthias Abend.  Cognitive Foundations Of Musical Pitch , 2016 .

[4]  Emilia Gómez Gutiérrez,et al.  Tonal description of music audio signals , 2006 .

[5]  Don Knox.  Book Review: Meinard Müller, Fundamentals of music processing , 2016 .

[6]  Meinard Müller,et al.  Matching Musical Themes based on noisy OCR and OMR input , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[7]  Peter Grosche,et al.  Audio Content-Based Music Retrieval , 2012, Multimodal Music Processing.

[8]  Falk Scholer,et al.  Searching Musical Audio Using Symbolic Queries , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[9]  Mark B. Sandler,et al.  Polyphonic Score Retrieval Using Polyphonic Audio Queries: A Harmonic Modeling Approach , 2003, ISMIR.

[10]  Masataka Goto,et al.  Spotting a Query Phrase from Polyphonic Music Audio Signals Based on Semi-supervised Nonnegative Matrix Factorization , 2014, ISMIR.

[11]  Peter Grosche,et al.  Toward characteristic audio shingles for efficient cross-version music retrieval , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[12]  Yvonne Feierabend.  A Dictionary Of Musical Themes , 2016 .

[13]  Anssi Klapuri,et al.  Query by humming of midi and audio using locality sensitive hashing , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[14]  Gerhard Widmer,et al.  Fast Identification of Piece and Score Position via Symbolic Fingerprinting , 2012, ISMIR.

[15]  Meinard Müller,et al.  A digital library framework for heterogeneous music collections: from document acquisition to cross-modal interaction , 2012, International Journal on Digital Libraries.

[16]  Meinard Müller,et al.  Towards Timbre-Invariant Audio Features for Harmony-Based Music , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[17]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[18]  Nicola Orio,et al.  An Efficient Identification Methodology for Improved Access to Music Heritage Collections , 2012, J. Multim..

[19]  Frans Wiering,et al.  Robust Segmentation and Annotation of Folk Song Recordings , 2009, ISMIR.

[20]  Meinard Müller,et al.  Chroma Toolbox: Matlab Implementations for Extracting Variants of Chroma-Based Audio Features , 2011, ISMIR.

[21]  Emilia Gómez,et al.  Tonal representations for music retrieval: from version identification to query-by-humming , 2012, International Journal of Multimedia Information Retrieval.

[22]  Daniel P. W. Ellis,et al.  Large-Scale Content-Based Matching of MIDI and Audio Files , 2015, ISMIR.

[23]  Emilia Gómez,et al.  Melody Extraction From Polyphonic Music Signals Using Pitch Contour Characteristics , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[24]  Masataka Goto,et al.  A chorus-section detecting method for musical audio signals , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[25]  Meinard Müller,et al.  Sheet Music-Audio Identification , 2009, ISMIR.