Mining Melodic Patterns in Large Audio Collections of Indian Art Music

Discovery of repeating structures in music is fundamental to its analysis, understanding and interpretation. We present a data-driven approach for the discovery of short-time melodic patterns in large collections of Indian art music. The approach first discovers melodic patterns within an audio recording and subsequently searches for their repetitions in the entire music collection. We compute similarity between melodic patterns using dynamic time warping (DTW). Furthermore, we investigate four different variants of the DTW cost function for rank refinement of the obtained results. The music collection used in this study comprises 1,764 audio recordings with a total duration of 365 hours. Over 13 trillion DTW distance computations are done for the entire dataset. Due to the computational complexity of the task, different lower bounding and early abandoning techniques are applied during DTW distance computation. An evaluation based on expert feedback on a subset of the dataset shows that the discovered melodic patterns are musically relevant. Several musically interesting relationships are discovered, yielding further scope for establishing novel similarity measures based on melodic patterns. The discovered melodic patterns can further be used in challenging computational tasks such as automatic raga recognition, composition identification and music recommendation.

[1]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[2]  Eamonn J. Keogh,et al.  Exact Discovery of Time Series Motifs , 2009, SDM.

[3]  Emilios Cambouropoulos,et al.  Musical Parallelism and Melodic Segmentation: : A Computational Approach , 2006 .

[4]  Célestin Deliège À propos de l’ouvrage de Lerdahl et Jackendoff : « A Generative Theory of Tonal Music » , 1983 .

[5]  Joe Cheri Ross,et al.  Detecting Melodic Motifs from Audio for Hindustani Classical Music , 2012, ISMIR.

[6]  Eamonn J. Keogh,et al.  Addressing Big Data Time Series: Mining Trillions of Time Series Subsequences Under Dynamic Time Warping , 2013, TKDD.

[7]  Ajay Srinivasamurthy,et al.  Corpora for Music Information Research in Indian Art Music , 2014, ICMC.

[8]  Jeremy Buhler,et al.  Finding motifs using random projections , 2001, RECOMB.

[9]  Nicholas Cook,et al.  A guide to musical analysis , 1987 .

[10]  W. Bas de Haas,et al.  Finding Repeated Patterns in Music: State of Knowledge, Challenges, Perspectives , 2013, CMMR.

[11]  Xavier Serra,et al.  Dunya: a system for browsing audio music collections exploiting cultural context , 2013, ISMIR 2013.

[12]  Hema A. Murthy,et al.  Classification of Melodic Motifs in Raga Music with Time-series Matching , 2014 .

[13]  Xavier Serra,et al.  Landmark Detection in Hindustani Music Melodies , 2014, ICMC.

[14]  Hema A. Murthy,et al.  Motif Spotting in an Alapana in Carnatic Music , 2013, ISMIR.

[15]  Darrell Conklin,et al.  Discovery of distinctive patterns in music , 2010, Intell. Data Anal..

[16]  Cormac Herley,et al.  ARGOS: automatically extracting repeating objects from multimedia streams , 2006, IEEE Transactions on Multimedia.

[17]  Eamonn J. Keogh,et al.  A symbolic representation of time series, with implications for streaming algorithms , 2003, DMKD '03.

[18]  Emilia Gómez,et al.  Melody Extraction From Polyphonic Music Signals Using Pitch Contour Characteristics , 2012, IEEE Transactions on Audio, Speech, and Language Processing.

[19]  Mikhail J. Atallah,et al.  Pattern Matching Image Compression: Algorithmic and Empirical Results , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Xavier Serra,et al.  Creating Research Corpora for the Computational Study of Music: the case of the CompMusic Project , 2014, Semantic Audio.

[21]  Xavier Serra,et al.  Essentia: An Audio Analysis Library for Music Information Retrieval , 2013, ISMIR.

[22]  Xavier Serra,et al.  Automatic Tonic Identification in Indian Art Music: Approaches and Evaluation , 2014 .

[23]  Dennis Shasha,et al.  Warping indexes with envelope transforms for query by humming , 2003, SIGMOD '03.

[24]  Anssi Klapuri,et al.  State of the Art Report: Audio-Based Music Structure Analysis , 2010, ISMIR.

[25]  S. Holm A Simple Sequentially Rejective Multiple Test Procedure , 1979 .

[26]  Eric O. Postma,et al.  Texton-based analysis of paintings , 2010, Optical Engineering + Applications.

[27]  S. Chiba,et al.  Dynamic programming algorithm optimization for spoken word recognition , 1978 .

[28]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[29]  Olivier Lartillot,et al.  Multi-Dimensional motivic pattern extraction founded on adaptive redundancy filtering , 2005 .

[30]  Florian Krebs,et al.  Bridging the Audio-Symbolic Gap: The Discovery of Repeated Note Content Directly from Polyphonic Music Audio , 2014, Semantic Audio.

[31]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[32]  Xavier Serra,et al.  ESSENTIA: an open-source library for sound and music analysis , 2013, ACM Multimedia.

[33]  R. Jackendoff,et al.  A Generative Theory of Tonal Music , 1985 .

[34]  Ning Hu,et al.  Pattern Discovery Techniques for Music Audio , 2002, ISMIR.

[35]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[36]  W. B. Haas,et al.  Discovering repeated patterns in music : state of knowledge , challenges , perspectives , 2013 .

[37]  T. Viswanathan,et al.  Music in South India : the Karṇāṭak concert tradition and beyond : experiencing music, expressing culture , 2004 .