Mid-Level Music Melody Representation of Polyphonic Audio for Query-by-Humming System

Recently a great attention is paid to content-based multimedia retrieval that enables users to find and locate audio-visual materials according to the intrinsic characteristics of the target. Query-by-humming (QBH) is also an application that makes retrieval based on major characteristics of music, that is, "melody". There have been some researches on QBH system, most of which are to retrieve music from symbolic music data by humming query. However, when the usability of technology is taken into consideration, retrieval of music in the form of polyphonic raw audio would be more useful and needed in the applications such as internet music search or music juke box, where the music data is stored not in symbolic form but in raw digital audio signal because such music data is more natural format for consumption. Our focus is on the realization of query-by-humming technology for an easy-to-use application, which entails full automation of all the processes of the system, including melody information extraction from polyphonic raw audio. In our system, melody feature of music database and humming is not represented by distinct note information but by the probability of note occurrence. Similarity is then measured between the melody features of humming and music data using DP matching method. This paper presents developed algorithms and experimental results for key steps of QBH system including the melody feature extraction method from polyphonic audio and humming, their representation for matching, and matching method between represented melody information from polyphonic audio and humming.