Dynamic Spectral Envelope Modeling for Timbre Analysis of Musical Instrument Sounds

We present a computational model of musical instrument sounds that focuses on capturing the dynamic behavior of the spectral envelope. A set of spectro-temporal envelopes belonging to different notes of each instrument are extracted by means of sinusoidal modeling and subsequent frequency interpolation, before being subjected to principal component analysis. The prototypical evolution of the envelopes in the obtained reduced-dimensional space is modeled as a nonstationary Gaussian Process. This results in a compact representation in the form of a set of prototype curves in feature space, or equivalently of prototype spectro-temporal envelopes in the time-frequency domain. Finally, the obtained models are successfully evaluated in the context of two music content analysis tasks: classification of instrument samples and detection of instruments in monaural polyphonic mixtures.

[1]  Kristoffer Jensen,et al.  The timbre model , 2002 .

[2]  Hani Yehia,et al.  Timbre Classification Of A Single Musical Instrument , 2004, ISMIR.

[3]  John Backus,et al.  The Acoustical Foundations of Music , 1970 .

[4]  Ian Kaminskyj,et al.  Automatic source identification of monophonic musical instrument sounds , 1995, Proceedings of ICNN'95 - International Conference on Neural Networks.

[5]  Masataka Goto,et al.  Musical instrument identification based on F0-dependent multivariate normal distribution , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[6]  Thomas Sikora,et al.  Monaural Source Separation from Musical Mixtures Based on Time-Frequency Timbre Models , 2007, ISMIR.

[7]  Stan Davis,et al.  Comparison of Parametric Representations for Monosyllabic Word Recognition in Continuously Spoken Se , 1980 .

[8]  Mathieu Lagrange,et al.  Polyphonic Instrument Recognition Using Spectral Clustering , 2007, ISMIR.

[9]  S. Schwerman,et al.  The Physics of Musical Instruments , 1991 .

[10]  Michael A. Casey.  Sound Classification and Similarity Tools , 2002 .

[11]  J. Grey.  Multidimensional perceptual scaling of musical timbres. , 1977, The Journal of the Acoustical Society of America.

[12]  William L. Martens,et al.  Perceptual Evaluation of Principal-Component-Based Synthesis of Musical Timbres , 1995 .

[13]  Keld K. Jensen,et al.  Timbre Models of Musical Sounds , 1999 .

[14]  Bozena Kostek,et al.  Musical instrument classification and duet analysis employing music information retrieval techniques , 2004, Proceedings of the IEEE.

[15]  Masataka Goto,et al.  RWC Music Database: Music genre database and musical instrument sound database , 2003, ISMIR.

[16]  Christophe Hourdin,et al.  A Multidimensional Scaling Analysis of Musical Instruments' Time-Varying Spectra , 1997 .

[17]  Emmanuel Vincent,et al.  Instrument-Specific Harmonic Atoms for Mid-Level Music Representation , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[18]  Gaël Richard,et al.  Instrument recognition in polyphonic music , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[19]  Perfecto Herrera-Boyer,et al.  Automatic Classification of Musical Instrument Sounds , 2003 .

[20]  J C Brown,et al.  Feature dependence in the automatic identification of musical woodwind instruments. , 2001, The Journal of the Acoustical Society of America.

[21]  Xavier Rodet,et al.  The Significance of the Non-Harmonic "Noise" Versis the Harmonic Series for Musical Instrument Recognition , 2006, ISMIR.

[22]  Paolo Prandoni,et al.  Sonological models for timbre characterization , 1997 .

[23]  Andrew Horner.  Author's Reply to Comments on 'A Simplified Wavetable Matching Method Using Combinatorial Basis Spectra Selection' , 2001 .

[24]  Xavier Rodet,et al.  An Accurate Timbre Model for Musical Instruments and its Application to Classification , 2006 .

[25]  Axel Röbel,et al.  Polyphonic musical instrument recognition based on a dynamic model of the spectral envelope , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.