Basis-Function Modeling of Loudness Variations in Ensemble Performance

This paper describes a computational model of loudness variations in expressive ensemble performance. The model predicts and explains the continuous variation of loudness as a function of information extracted automatically from the written score. Although such models have been proposed for expressive performance in solo instruments, this is (to the best of our knowledge) the first attempt to define a model for expressive performance in ensembles. To that end, we extend an existing model that was designed to model expressive piano performances, and describe the additional steps necessary for the model to deal with scores of arbitrary instrumentation, including orchestral scores. We test both linear and non-linear variants of the extended model n a data set of audio recordings of symphonic music, in a leave-one-out setting. The experiments reveal that the most successful model variant is a recurrent, non-linear model. Even if the accuracy of the predicted loudness varies from one recording to another, in several cases the model explains well over 50% of the variance in loudness.

[1]  Neil P. McAngus Todd,et al.  A computational model of rubato , 1989 .

[2]  Giovanni De Poli,et al.  An Abstract Control Space for Communication of Sensory Expressive Intentions in Music Performance , 2003 .

[3]  J. Sloboda.  The Communication of Musical Metre in Piano Performance , 1983 .

[4]  Gerhard Widmer,et al.  Machine Discoveries: A Few Simple, Robust Local Expression Principles , 2002 .

[5]  Eric Clarke,et al.  Expression and communication in musical performance , 1991 .

[6]  Razvan Pascanu,et al.  How to Construct Deep Recurrent Neural Networks , 2013, ICLR.

[7]  Alfred Binet,et al.  Recherches graphiques sur la musique , 1895 .

[8]  Emery Schubert,et al.  Expressiveness in music performance : empirical approaches across styles and cultures , 2014 .

[9]  Erwin Schoonderwaldt,et al.  Extraction of bowing parameters from violin performance combining motion capture and sensors. , 2009, The Journal of the Acoustical Society of America.

[10]  Thomas L. Rhea,et al.  Evolution of the Keyboard Interface: The Bosendorfer 290 SE Recording Piano and the Moog Multiply-Touch-Sensitive Keyboards , 1990 .

[11]  Jordi Bonada,et al.  Low-Latency Instrument Separation in Polyphonic Audio Using Timbre Models , 2012, LVA/ICA.

[12]  Giovanni De Poli.  Analysis and Modeling of Expressive Intentions in Music Performance , 2003 .

[13]  Gerhard Widmer,et al.  Automatic Page Turning for Musicians via Real-Time Machine Listening , 2008, ECAI.

[14]  Roberto Bresin,et al.  Artificial neural networks based models for automatic performance of musical scores , 1998 .

[15]  G. Widmer,et al.  A method to determine the contribution of annotated performance directives in music performances , 2011 .

[16]  Gerhard Widmer,et al.  Discovering simple rules in complex data: A meta-learning algorithm and some surprising musical discoveries , 2003, Artif. Intell..

[17]  P. Juslin.  Emotional Communication in Music Performance: A Functionalist Perspective and Some Data , 1997 .

[18]  Carlos Eduardo Cancino Chacón,et al.  Temporal Dependencies in the Expressive Timing of Classical Piano Performances , 2017 .

[19]  P. Keller.  Ensemble performance: Interpersonal alignment of musical expression , 2013 .

[20]  Gerhard Widmer,et al.  Computational Models of Expressive Music Performance: The State of the Art , 2004 .

[21]  Roger B. Dannenberg,et al.  Spectral Learning for Expressive Interactive Ensemble Music Performance , 2015, ISMIR.

[22]  Carlos Eduardo Cancino Chacón,et al.  An Evaluation of Score Descriptors Combined with Non-linear Models of Expressive Dynamics in Music , 2015, Discovery Science.

[23]  Gerhard Widmer,et al.  Automatic Alignment of Music Performances with Structural Differences , 2013, ISMIR.

[24]  Robert O. Gjerdingen,et al.  The Psychology of Music , 1972 .

[25]  W. Goebl,et al.  Measurement setup for articulatory transient differences in woodwind performance , 2013 .

[26]  Shin-ichi Maeda,et al.  Gaussian Process Regression for Rendering Music Performance , 2008 .

[27]  Is He Playing Solo or within an Ensemble? How the Context, Visual Information, and Expertise May Impact upon the Perception of Musical Expressivity , 2014, Perception.

[28]  Hervé Bourlard,et al.  Generalization and Parameter Estimation in Feedforward Netws: Some Experiments , 1989, NIPS.

[29]  Jordi Janer,et al.  Improving Score-Informed Source Separation for Classical Music through Note Refinement , 2015, ISMIR.

[30]  W Goebl,et al.  Melody lead in piano performance: expressive device or artifact? , 2001, The Journal of the Acoustical Society of America.

[31]  J. Sundberg,et al.  Overview of the KTH rule system for musical performance. , 2006 .

[32]  W. L. Windsor,et al.  Expressive Timing and Dynamics in Real and Artificial Musical Performances: Using an Algorithm as an Analytical Tool , 1997 .

[33]  C. Palmer,et al.  Synchronization of Timing and Motion 435 , 2022 .

[34]  Giovanni De Poli.  Analysis and modeling of expressive intentions in music performance. , 2003, Annals of the New York Academy of Sciences.

[35]  David P. Helmbold,et al.  Modeling, analyzing, and synthesizing expressive piano performance with graphical models , 2006, Machine Learning.

[36]  W. Goebl,et al.  Context-specific effects of musical expertise on audiovisual integration , 2014, Front. Psychol..

[37]  J. Sloboda,et al.  Music and emotion: Theory and research , 2001 .

[38]  Alex Graves,et al.  Generating Sequences With Recurrent Neural Networks , 2013, ArXiv.

[39]  Florian Krebs,et al.  An Assessment of Learned Score Features for Modeling Expressive Dynamics in Music , 2014, IEEE Transactions on Multimedia.

[40]  Tapio Lokki,et al.  Anechoic recording system for symphony orchestra , 2008 .

[41]  Gerhard Widmer,et al.  Linear Basis Models for Prediction and Analysis of Musical Expression , 2012 .