Using AI and machine learning to study expressive music performance: project survey and first report

This article presents a longdterm interddisciplinary research project situated at the intersection of the scientific disciplines of Musicology and Artificial Intelligence. The goal is to develop AI, and in particular machine learning and data mining, methods to study the complex phenomenon of expressive music performance. Formulating formal, quantitative models of expressive performance is one of the big open research problems in contemporary (empirical and cognitive) musicology. Our project develops a new direction in this field: we use inductive learning techniques to discover general and valid expression principles from (large amounts of) real performance data. The project is currently starting its third year and is planned to continue for at least four more years.In the following, we explain the basic notions of expressive music performance, and why this is such a central phenomenon in music. We present the general research framework of the project, and discuss the various challenges and research opportunities that emerge in this framework. We then briefly describe the current state of the project and list the main achievements made so far. In the rest of the paper, we discuss in more detail one particular data mining approach (including a new algorithm for learning characterisation rules) that we have developed recently. Preliminary experimental results demonstrate that this algorithm can discover very general and robust expression principles, some of which actually constitute novel discoveries from a musicological viewpoint.

[1]  Johan Sundberg,et al.  Common Secrets of Musicians and Listeners - An analysis-by-synthesis Study of Musical Performance , 1991 .

[2]  Thomas G. Dietterich.  Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[3]  Emilios Cambouropoulos,et al.  From MIDI to Traditional Musical Notation , 2000 .

[4]  Gerhard Widmer,et al.  Modeling the rational basis of musical expression , 1995 .

[5]  Gerhard Widmer,et al.  Large-scale Induction of Expressive Performance Rules: First Quantitative Results , 2000, ICMC.

[6]  Gerhard Widmer.  Combining Knowledge-Based and Instance-Based Learning to Exploit Qualitative Knowledge , 1993, Informatica.

[7]  Gerhard Widmer,et al.  Learning expressive performance: The structure‐level approach , 1996 .

[8]  L. H. Shaffer,et al.  Metre and rhythm in piano playing , 1985, Cognition.

[9]  B. Repp.  Diversity and commonality in music performance: an analysis of timing microstructure in Schumann's "Träumerei". , 1992, The Journal of the Acoustical Society of America.

[10]  S. Dixon,et al.  Analysis of tempo classes in performances of Mozart sonatas , 2001 .

[11]  Gerhard Widmer,et al.  Discovering Strong Principles of Expressive Music Performance with the PLCG Rule Learning Strategy , 2001, ECML.

[12]  B. Repp.  The Aesthetic Quality of a Quantitatively Average Music Performance: Two Preliminary Experiments , 1997 .

[13]  E. Narmour.  The Analysis and Cognition of Melodic Complexity: The Implication-Realization Model , 1992 .

[14]  W Goebl,et al.  Melody lead in piano performance: expressive device or artifact? , 2001, The Journal of the Acoustical Society of America.

[15]  Roberto Bresin.  Virtual Virtuosity Studies in Automatic Music Performance , 2000 .

[16]  Simon Dixon,et al.  Beat Tracking with Musical Knowledge , 2000, ECAI.

[17]  Stephen Muggleton,et al.  Protein secondary structure prediction using logic-based machine learning , 1992 .

[18]  B. Repp.  A microcosm of musical expression. I. Quantitative analysis of pianists' timing in the initial measures of Chopin's Etude in E major. , 1998, The Journal of the Acoustical Society of America.

[19]  Daniel Dominic Sleator,et al.  Modeling Meter and Harmony: A Preference-Rule Approach , 1999, Computer Music Journal.

[20]  Neil P. McAngus Todd,et al.  Towards a cognitive theory of expression: The performance and perception of rubato , 1989 .

[21]  Simon Dixon.  An Interactive Beat Tracking and Visualisation System , 2001, ICMC.

[22]  Emilios Cambouropoulos.  Automatic Pitch Spelling: From Numbers to Sharps and Flats , 2001 .

[23]  Eric Clarke,et al.  Some Aspects of Rhythm and Expression in Performances of Erik Satie's "Gnossienne No. 5" , 1985 .

[24]  Simon Dixon,et al.  Automatic Extraction of Tempo and Beat From Expressive Performances , 2001 .

[25]  Lawrence Hunter,et al.  Artificial Intelligence and Molecular Biology , 1992, AI Mag..

[26]  Emilios Cambouropoulos,et al.  Melodic Cue Abstraction, Similarity, and Category Formation: A Formal Model , 2001 .

[27]  Anders Friberg,et al.  A Quantitative Rule System for Musical Performance , 1995 .

[28]  Caroline Palmer,et al.  Timing in skilled piano performance , 1989 .

[29]  Emilios Cambouropoulos,et al.  The Local Boundary Detection Model (LBDM) and its Application in the Study of Expressive Timing , 2001, ICMC.

[30]  Gerhard Widmer,et al.  Production of staccato articulation in Mozart sonatas played on a grand piano. : Preliminary results , 2000 .

[31]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[32]  Gerhard Widmer,et al.  A clustering algorithm for melodic analysis , 1999 .

[33]  Simon Dixon,et al.  A Lightweight Multi-agent Musical Beat Tracking System , 2000, PRICAI.

[34]  Gerhard Widmer,et al.  Automated Motivic Analysis via Melodic Clustering , 2000 .

[35]  Gerhard Widmer.  What Is It That Makes It a Horowitz? Empirical Musicology via Machine Learning , 1996, ECAI.

[36]  N. Todd.  The dynamics of dynamics: A model of musical expression , 1992 .

[37]  Caroline Palmer,et al.  On the Assignment of Structure in Music Performance , 1996 .

[38]  Richard A. Lewis,et al.  Drug design by machine learning: the use of inductive logic programming to model the structure-activity relationships of trimethoprim analogues binding to dihydrofolate reductase. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[39]  Gerhard Widmer,et al.  Machine Discoveries: A Few Simple, Robust Local Expression Principles , 2002 .

[40]  Sholom M. Weiss,et al.  Rule-based Machine Learning Methods for Functional Prediction , 1995, J. Artif. Intell. Res..