Name that tune: A pilot study in finding a melody from a sung query

We have created a system for music search and retrieval. A user sings a theme from the desired piece of music. The sung theme (query) is converted into a sequence of pitch-intervals and rhythms. This sequence is compared to musical themes (targets) stored in a data-base. The top pieces are returned to the user in order of similarity to the sung theme. We describe, in detail, two different approaches to measuring similarity between database themes and the sung query. In the first, queries are compared to database themes using standard string-alignment algorithms. Here, similarity between target and query is determined by edit cost. In the second approach, pieces in the database are represented as hidden Markov models (HMMs). In this approach, the query is treated as an observation sequence and a target is judged similar to the query if its HMM has a high likelihood of generating the query. In this article we report our approach to the construction of a target database of themes, encoding, and transcription of user queries, and the results of preliminary experimentation with a set of sung queries. Our experiments show that while no approach is clearly superior to the other system, string matching has a slight advantage. Moreover, neither approach surpasses human performance.

[1]  Miller Puckette,et al.  Score Following in Practice , 1992, ICMC.

[2]  Adriane Durey,et al.  Melody Spotting Using Hidden Markov Models , 2001, ISMIR.

[3]  Matti Karjalainen,et al.  A computationally efficient multipitch analysis model , 2000, IEEE Trans. Speech Audio Process..

[4]  William P. Birmingham,et al.  The MusArt Music-Retrieval System: An Overview: D-lib Magazine , 2002 .

[5]  J. Stephen Downie,et al.  Evaluation of a simple and effective music information retrieval method , 2000, SIGIR '00.

[6]  Jeremy Pickens.  A Comparison of Language Modeling and Probabilistic Text Information Retrieval Approaches to Monophonic Music Retrieval , 2000, ISMIR.

[7]  Justin Zobel,et al.  Melodic matching techniques for large music databases , 1999, MULTIMEDIA '99.

[8]  Ian H. Witten,et al.  Towards the digital music library: tune retrieval from acoustic input , 1996, DL '96.

[9]  Stefan M. Rüger,et al.  A Comparative and Fault-tolerance Study of the Use of N-grams with Polyphonic Music , 2002, ISMIR.

[10]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[11]  Marc Leman,et al.  An Auditory Model Based Transcriber of Singing Sequences , 2002, ISMIR.

[12]  William P. Birmingham,et al.  Improved Score Following for Acoustic Performances , 2002, ICMC.

[13]  Roger B. Dannenberg,et al.  An On-Line Algorithm for Real-Time Accompaniment , 1984, ICMC.

[14]  Yuen-Hsien Tseng,et al.  Content-based retrieval for music collections , 1999, SIGIR '99.

[15]  William P. Birmingham,et al.  Johnny Can't Sing: A Comprehensive Error Model for Sung Music Queries , 2002, ISMIR.

[16]  William P. Birmingham,et al.  The MusArt Music-Retrieval System: An Overview , 2002, D-Lib Magazine.

[17]  William P. Birmingham,et al.  HMM-based musical query retrieval , 2002, JCDL '02.

[18]  Andrew D. Sterian,et al.  Model-based segmentation of time-frequency images for musical transcription. , 1999 .

[19]  William P. Birmingham,et al.  Encoding Timing Information for Musical Query Matching , 2002, ISMIR.

[20]  S. B. Needleman,et al.  A general method applicable to the search for similarities in the amino acid sequence of two proteins. , 1970, Journal of molecular biology.

[21]  Michael Clausen,et al.  PROMS: A Web-based Tool for Searching in Polyphonic Music , 2000, ISMIR.

[22]  Ning Hu,et al.  A Probabilistic Model of Melodic Similarity , 2002, ICMC.

[23]  William P. Birmingham,et al.  Following a Musical Performance from a Partially Specified Score , 2001 .

[24]  Eleanor Selfridge-Field,et al.  Melodic Similarity : concepts, procedures, and applications , 1998 .

[25]  Sean R. Eddy,et al.  Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids , 1998 .

[26]  O. Gotoh.  An improved algorithm for matching biological sequences. , 1982, Journal of molecular biology.

[27]  Holger H. Hoos,et al.  GUIDO/MIR - an Experimental Musical Information Retrieval System based on GUIDO Music Notation , 2001, ISMIR.

[28]  Andreas Kornstädt,et al.  Themefinder: A web-based melodic search tool , 1998 .

[29]  Avelyn Davidson,et al.  Happy Birthday , 1997 .

[30]  Christopher Raphael,et al.  Automatic Segmentation of Acoustic Musical Signals Using Hidden Markov Models , 1999, IEEE Trans. Pattern Anal. Mach. Intell..