A Concept for Using Combined Multimodal Queries in Digital Music Libraries

In this paper, we propose a concept for using combined multimodal queries in the context of digital music libraries. Whereas usual mechanisms for content-based music retrieval only consider a single query mode, such as query-by-humming, full-text lyrics-search or query-by-example using short audio snippets, our proposed concept allows to combine those different modalities into one integrated query. Our particular contributions consist of concepts for query formulation, combined content-based retrieval and presentation of a suitably ranked result list. The proposed concepts have been realized within the context of the PROBADO Music Repository and allow for music retrieval based on combining full-text lyrics search and score-based query-by-example search.

[1]  Stuart Macdonald,et al.  User Engagement in Research Data Curation , 2009, ECDL.

[2]  Meinard Müller,et al.  Automatic Mapping of Scanned Sheet Music to Audio Recordings , 2008, ISMIR.

[3]  Donald Byrd,et al.  Prospects for Improving OMR with Multiple Recognizers , 2006, ISMIR.

[4]  Ian H. Witten,et al.  Managing gigabytes 2nd edition , 1999 .

[5]  George Tzanetakis,et al.  Polyphonic audio matching and alignment for music retrieval , 2003, 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No.03TH8684).

[6]  Meinard Müller,et al.  Automated Synchronization of Scanned Sheet Music with Audio Recordings , 2007, ISMIR.

[7]  Alistair Moffat,et al.  Methodologies for distributed information retrieval , 1998, Proceedings. 18th International Conference on Distributed Computing Systems (Cat. No.98CB36183).

[8]  Meinard Müller,et al.  A Framework for Managing Multimodal Digitized Music Collections , 2008, ECDL.

[9]  Meinard Müller,et al.  Lyrics-Based Audio Retrieval and Multimodal Navigation in Music Collections , 2007, ECDL.

[10]  Hans-Jürgen Appelrath,et al.  PROBADO - A Generic Repository Integration Framework , 2007, ECDL.

[11]  Hans-Peter Kriegel,et al.  Content-Based Image Retrieval Using Multiple Representations , 2004, KES.

[12]  Andreas Henrich,et al.  Combining Multimedia Retrieval and Text Retrieval to Search Structured Documents in Digital Libraries , 2000, DELOS.

[13]  Meinard Müller,et al.  Efficient Index-Based Audio Matching , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[14]  Gregory H. Wakefield,et al.  Audio thumbnailing of popular music using chroma-based representations , 2005, IEEE Transactions on Multimedia.