Automatic Transcription of Polyphonic Piano Music Using Genetic Algorithms, Adaptive Spectral Envelope Modeling, and Dynamic Noise Level Estimation

This paper presents a new method for multiple fundamental frequency (F0) estimation on piano recordings. We propose a framework based on a genetic algorithm in order to analyze the overlapping overtones and search for the most likely F0 combination. The search process is aided by adaptive spectral envelope modeling and dynamic noise level estimation: while the noise is dynamically estimated, the spectral envelope of previously recorded piano samples (internal database) is adapted in order to best match the piano played on the input signals and aid the search process for the most likely combination of F0s. For comparison, several state-of-the-art algorithms were run across various musical pieces played by different pianos and then compared using three different metrics. The proposed algorithm ranked first place on Hybrid Decay/Sustain Score metric, which has better correlation with the human hearing perception and ranked second place on both onset-only and onset–offset metrics. A previous genetic algorithm approach is also included in the comparison to show how the proposed system brings significant improvements on both quality of the results and computing time.

[1]  Soledad Torres-Guijarro,et al.  Multiple Piano Note Identification Using a Spectral Matching Method with Derived Patterns , 2005 .

[2]  William E. Hart,et al.  Memetic Evolutionary Algorithms , 2005 .

[3]  B. P. Bogert,et al.  The quefrency analysis of time series for echoes : cepstrum, pseudo-autocovariance, cross-cepstrum and saphe cracking , 1963 .

[4]  S. Schwerman,et al.  The Physics of Musical Instruments , 1991 .

[5]  Peter Norvig,et al.  Artificial intelligence - a modern approach, 2nd Edition , 2003, Prentice Hall series in artificial intelligence.

[6]  David Lu.  Automatic Music Transcription Using Genetic Algorithms and Electronic Synthesis David Lu ! ! April , 2006 .

[7]  Marc Leman,et al.  Automatic Harmonic Description of Musical Signals Using Schema-based Chord Decomposition , 1999 .

[8]  Jerry D. Gibson,et al.  COMPARISON OF DISTANCE MEASURES IN DISCRETE SPECTRAL MODELING , 2000 .

[9]  Keith D. Martin,et al.  A Blackboard System for Automatic Transcription of Simple Polyphonic Music , 1996 .

[10]  Simon J. Godsill,et al.  Polyphonic pitch tracking using joint Bayesian estimation of multiple frame parameters , 1999, Proceedings of the 1999 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. WASPAA'99 (Cat. No.99TH8452).

[11]  D. E. Goldberg,et al.  Genetic Algorithms in Search, Optimization & Machine Learning , 1989 .

[12]  Guillermo García.  A Genetic Search Technique for Polyphonic Pitch Detection , 2001, ICMC.

[13]  Mert Bay,et al.  The Music Information Retrieval Evaluation eXchange: Some Observations and Insights , 2010, Advances in Music Information Retrieval.

[14]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[15]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[16]  Francisco Fernández de Vega,et al.  Electronic synthesis using genetic algorithms for automatic music transcription , 2007, GECCO '07.

[17]  Luis Gustavo Martins,et al.  A computational framework for sound segregation in music signals , 2008 .

[18]  Simon J. Godsill,et al.  Bayesian Graphical Models for Polyphonic Pitch Tracking , 1999 .

[19]  Emmanuel Vincent,et al.  Adaptive Harmonic Spectral Decomposition for Multiple Pitch Estimation , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[20]  David Barber,et al.  A generative model for music transcription , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[21]  Jasmin Komic,et al.  Harmonic Mean , 2011, International Encyclopedia of Statistical Science.

[22]  Matija Marolt,et al.  A connectionist approach to automatic transcription of polyphonic piano music , 2004, IEEE Transactions on Multimedia.

[23]  Harvey Fletcher,et al.  Quality of Piano Tones , 1962 .

[24]  Roland Badeau,et al.  Blind Signal Decompositions for Automatic Transcription of Polyphonic Music: NMF and K-SVD on the Benchmark , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[25]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[26]  Roland Badeau,et al.  Automatic transcription of piano music based on HMM tracking of jointly-estimated pitches , 2008, 2008 16th European Signal Processing Conference.

[27]  Anssi Klapuri,et al.  Multipitch Analysis of Polyphonic Music and Speech Signals Using an Auditory Model , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[28]  Valentin Emiya.  Transcription automatique de la musique de piano , 2008 .

[29]  Roland Badeau,et al.  Multipitch Estimation of Piano Sounds Using a New Probabilistic Spectral Smoothness Principle , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[30]  Kunio Kashino,et al.  Organization of Hierarchical Perceptual Sounds: Music Scene Analysis with Autonomous Processing Modules and a Quantitative Information Integration Mechanism , 1995, IJCAI.

[31]  S. Dixon.  ONSET DETECTION REVISITED , 2006 .

[32]  M.P. Ryynanen,et al.  Polyphonic music transcription using note event modeling , 2005, IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, 2005..

[33]  N. Fonseca,et al.  A Genetic Algorithm Approach with Harmonic Structure Evolution for Polyphonic Music Transcription , 2008, 2008 IEEE International Symposium on Signal Processing and Information Technology.

[34]  Axel Röbel,et al.  Multiple Fundamental Frequency Estimation and Polyphony Inference of Polyphonic Music Signals , 2010, IEEE Transactions on Audio, Speech, and Language Processing.

[35]  Emmanuel Vincent,et al.  Harmonic and inharmonic Nonnegative Matrix Factorization for Polyphonic Pitch transcription , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[36]  G. Reis,et al.  Genetic Algorithm Approach to Polyphonic Music Transcription , 2007, 2007 IEEE International Symposium on Intelligent Signal Processing.

[37]  J. Stephen Downie,et al.  The music information retrieval evaluation exchange (2005-2007): A window into music information retrieval research , 2008 .

[38]  Aníbal Ferreira,et al.  Measuring music transcription results based on a hybrid decay/sustain evaluation , 2009 .

[39]  John Holland,et al.  Adaptation in Natural and Artificial Sys-tems: An Introductory Analysis with Applications to Biology , 1975 .

[40]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[41]  Matija Marolt.  Networks of Adaptive Oscillators for Partial Tracking and Transcription of Music Recordings , 2004 .

[42]  Christopher Raphael,et al.  Automatic Transcription of Piano Music , 2002, ISMIR.