Timbre models of musical sounds - from the model of one sound to the model of one instrument

This work involves the analysis of musical instrument sounds, the creation of timbre models, the estimation of the parameters of the timbre models and the analysis of the timbre model parameters. The timbre models are found by studying the literature of auditory perception, and by studying the gestures of music performance. Some of the important results from this work are an improved fundamental frequency estimator, a new envelope analysis method, and simple intuitive models for the sound of musical instruments. Furthermore a model for the spectral envelope is introduced in this work. A new function, the brightness creation function, is introduced in the spectral envelope model. The timbre model is used to analyze the evolution of the different timbre parameters when the fundamental frequency is changed, but also for different intensity, tempo, or style. The main results from this analysis are that brightness rises with frequency, but nevertheless the fundamental has almost all amplitude for the high notes. The attack and release times generally fall with frequency. It was found that only brightness and amplitude are affected by a change in intensity, and only the sustain and release times are affected when the tempo is changed. The different timbre models are also used for the classification of the sounds in musical instrument classes with very good results. Finally, listening tests have been performed, which assessed that the best timbre model has an acceptable sound quality. Resumé Dette arbejder omhandler analyse af musikinstrumenter, dannelse af modeller af musikinstrumenters klangfarve, estimering af klangfarve model parametre og analyse af modelparametrene. Klangfarvemodellerne er fundet ved at gennemgå lydperceptorisk litteratur, og ved at studere musikudøvelse. Nogle vigtige resultater fra dette arbejde er en forbedret fundamental frekvens estimator, en ny envelope analysemetode, og simple intuitive modeller af musiklyd. Desuden er en model af den spektrale envelope udviklet. I den forbindelse er en ny funktion for syntese af lyd med en given ‘brightness’ udviklet. Klangfarvemodellen er brugt til at analysere udviklingen af de forskellige klangfarveattributter, når fundamentalfrekvensen ændres, men også for forskellige intensiteter, tempi og stil. De vigtigste konklusioner fra dette arbejde er, at ‘brightness’ stiger med frekvens; men fundamentalen har alligevel næsten al amplitude for de høje toner. ‘Attack’ og ‘release’ tiderne falder med frekvensen. Af intensitetsog tempoændringer fandtes, at kun ‘brightness’ og amplituden ændres når intensiteten ændres, og at kun ‘sustain’ og ‘release’ tiderne ændres når tempoet ændres. De forskellige klangfarvemodeller er også brugt til klassifikation af lyd i instrumentklasser med meget godt resultat. Lytteforsøg godtgjorde, at den bedste klangfarvemodel har en acceptabel lydkvalitet. Résumé Ce travail traite l’analyse des sons musicaux, la création des modèles de timbre, l’estimation des paramètres des modèles de timbre, ainsi que l’analyse des paramètres des modèles. Les modèles de timbre ont été trouvés dans la littérature de la perception auditive et en étudiant les gestes du musicien. Quelques résultats importants du travail présenté ici sont une estimation améliorée de la fréquence fondamentale. Une nouvelle méthode pour l’estimation des temps d’attaque et de relâchement a été developpée, ainsi que des modèles intuitifs de sons d’instrument de musique. Un nouveau modèle d’enveloppe spectrale a été défini, ainsi qu’une fonction qui donne un son avec la brillance indiquée. Les modèles de timbre sont utilisés pour l’analyse de l’évolution des paramètres des timbres en fonction de la fréquence fondamentale, de l’intensité, du tempo ou du style. Le résultat principal de cette analyse est que la brillance monte avec la fréquence, mais que la fondamentale a presque toute l’amplitude dans les aigüs. Les temps d’attaque et relâchement diminuent avec la fréquence fondamentale. Pour une variation de l’intensité, seul l’amplitude et la brillance sont affectées. Seuls les temps de maintien et relâchement changent avec le tempo. Le modèle de timbre est aussi utilisé pour la classification des sons dans des classes d’instruments avec de très bons resultats. Finalement, des tests d’ecoute de tous les modèles ont permis de conclure que le meilleur modèle de timbre possède une qualité de son acceptable. Acknowledgments First and foremost, my thanks go to Jens Arnspang, who has created the music informatics group at the Computer Science Department at the University of Copenhagen, and without whom this work would never have started. Jens accepted to be my supervisor and had the open mind to let me pursue my own directions, and detours. Secondly, my thanks go to the two members of the monitor group, Ivar Frounberg and Holger Rindel for insightful comments and feedback both in the musical and the technical domain. The comments from them helped keep a focus in my work, and inspired further improvements. This work has been financed by the Danish Technical Research Council whom I thank. My thanks go to all members, past or present, of the music informatics group. Special thanks go to Klaus Hansen for invaluable help. Fruitful discussions with Stefan Borum, Anders Møller and Esben Skovenberg have also been a great source of inspiration. Sincere thanks goes to the musicians who accepted to spend time to record sounds which is not music. The musical instrument sound database created with their help has been instrumental in this work. The judgments and comments from the members of the listening tests have also been a great help. My sincere thanks go to all the participants in the listening tests. Many helpful comments have also come from other groups in the computer science department. Special thanks to Ketil Perstrup, Kristian Pilgaard, Jon Sporring, Joachim Weickert, Peter Riber, Stig Skelboe, Knud Henriksen, Erik Frøkjær and Morten Hanehøj. The image group and notably the scale-space community have been a great source of inspiration. My thoughts go to everybody at DIKU who have made my stay here so pleasant. Part of the thesis work in Denmark is passed in a different research institution, as required by the Danish Ph.D. circular. I was very lucky to be accepted at the Groupe Informatique Musical at the Laboratoire Mecanique et Acoustique in Marseille, France. My sincere thanks go to Jean-Claude Risset for having accepted me in his group, and to Richard Kronland-Martinet and Philippe Guillemain for help and discussions. My stay at the groupe informatique musicale was made agreeable by the fruitful discussions and the nice atmosphere in the group. A final thanks goes to Carol Jensen, Thomas Jensen and all the members of my family, and especially to Alice, who hopefully will see more of Papa soon.