Modeling and control of expressiveness in music performance

Expression is an important aspect of music performance. It is the added value of a performance and is part of the reason that music is interesting to listen to and sounds alive. Understanding and modeling expressive content communication is important for many engineering applications in information technology. For example, in multimedia products, textual information is enriched by means of graphical and audio objects. In this paper, we present an original approach to modify the expressive content of a performance in a gradual way, both at the symbolic and signal levels. To this purpose, we discuss a model that applies a smooth morphing among performances with different expressive content, adapting the audio expressive character to the user's desires. Morphing can be realized with a wide range of graduality (from abrupt to very smooth), allowing adaptation of the system to different situations. The sound rendering is obtained by interfacing the expressiveness model with a dedicated postprocessing environment, which allows for the transformation of the event cues. The processing is based on the organized control of basic audio effects. Among the basic effects used, an original method for the spectral processing of audio is introduced.

[1]  Curtis Roads,et al.  The Computer Music Tutorial , 1996 .

[2]  Ramón López de Mántaras,et al.  An Interactive Case-Based Reasoning Approach for Generating Expressive Music , 2004, Applied Intelligence.

[3]  N. Todd.  The dynamics of dynamics: A model of musical expression , 1992 .

[4]  Anders Friberg,et al.  Performance Rules for Computer-Controlled Contemporary Keyboard Music , 1991 .

[5]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[6]  Barry Vercoe,et al.  Structured audio: creation, transmission, and rendering of parametric sound representations , 1998, Proc. IEEE.

[7]  Seiji Inokuchi,et al.  Extraction of Musical Performance Rules Using a Modified Algorithm of Multiple Regression Analysis , 2000, ICMC.

[8]  O. Cappé,et al.  Regularized estimation of cepstrum envelope from discrete frequency points , 1995, Proceedings of 1995 Workshop on Applications of Signal Processing to Audio and Accoustics.

[9]  Giovanni De Poli,et al.  A fuzzy approach to performance rules , 1995 .

[10]  Yoshinori Sagisaka,et al.  Speech spectrum conversion based on speaker interpolation and multi-functional representation with weighting by radial basis function networks , 1995, Speech Commun..

[11]  Xavier Serra,et al.  Musical Sound Modeling with Sinusoids plus Noise , 1997 .

[12]  Gerhard Widmer,et al.  Learning expressive performance: The structure‐level approach , 1996 .

[13]  Carlo Drioli,et al.  Audio Morphing Different Expressive Intentions for Multimedia Systems , 2000, IEEE Multim..

[14]  Anders Friberg,et al.  Emotional Coloring of Computer-Controlled Music Performances , 2000, Computer Music Journal.

[15]  Johan Sundberg,et al.  Generating Musical Performances with Director Musices , 2000, Computer Music Journal.

[16]  Manfred Clynes,et al.  Microstructural musical linguistics: composers' pulses are liked most by the best musicians , 1995, Cognition.

[17]  Sergio Canazza,et al.  Perceptual Analysis of the Musical Expressive Intention in a Clarinet Performance , 1996, Joint International Conference on Cognitive and Systematic Musicology.

[18]  N. Todd.  A Model of Expressive Timing in Tonal Music , 1985 .

[19]  George N. Votsis,et al.  Emotion recognition in human-computer interaction , 2001, IEEE Signal Process. Mag..

[20]  C. Palmer.  Music performance. , 1997, Annual review of psychology.

[21]  Patrik N. Juslin,et al.  Can results from studies of perceived expression in musical performances be generalized across response formats , 1997 .

[22]  Manfred Clynes,et al.  Some Guidelines for the Synthesis and Testing of Pulse Microstructure in Relation to Musical Meaning , 1990 .

[23]  Thomas F. Quatieri,et al.  Speech analysis/Synthesis based on a sinusoidal representation , 1986, IEEE Trans. Acoust. Speech Signal Process..

[24]  Gerhard Widmer,et al.  Large-scale Induction of Expressive Performance Rules: First Quantitative Results , 2000, ICMC.

[25]  Giovanni De Poli,et al.  Music score interpretation using a multilevel knowledge base , 1990 .

[26]  Giovanni De Poli,et al.  Note‐by‐note analysis of the influence of expressive intentions and musical structure in violin performance* , 1998 .

[27]  N. Todd.  The kinematics of musical expression , 1995 .

[28]  Johan Sundberg,et al.  How can music be expressive? , 1993, Speech Commun..

[29]  J. Sloboda,et al.  Music and emotion: Theory and research , 2001 .

[30]  W. J. Pielemeier,et al.  Time-frequency analysis of musical signals , 1996, Proc. IEEE.

[31]  Xavier Serra,et al.  SaxEx: a case-based reasoning system for generating expressive musical performances , 1998, ICMC.

[32]  Alf Gabrielsson,et al.  Expressive Intention and Performance , 1995 .

[33]  Nicola Orio,et al.  A Parametric Model of Expressiveness in Musical Performance Based on Perceptual and Acoustic Analysis , 1999, ICMC.

[34]  Takenobu Tokunaga,et al.  A Case Based Approach to the Generation of Musical Expression , 1999, IJCAI.

[35]  Carlo Drioli.  Radial Basis Function Networks for Conversion of Sound Spectra , 2001, EURASIP J. Adv. Signal Process..

[36]  Seiji Inokuchi,et al.  Learning performance rules in a music interpretation system , 1993, Comput. Humanit..

[37]  Simon Haykin,et al.  Neural Networks: A Comprehensive Foundation , 1998 .

[38]  B. Repp.  Patterns of expressive timing in performances of a Beethoven minuet by nineteen famous pianists. , 1990 .

[39]  Roberto Bresin,et al.  Artificial neural networks based models for automatic performance of musical scores , 1998 .

[40]  E. Zwicker,et al.  Analytical expressions for critical‐band rate and critical bandwidth as a function of frequency , 1980 .

[41]  B. Repp.  Diversity and commonality in music performance: an analysis of timing microstructure in Schumann's "Träumerei". , 1992, The Journal of the Acoustical Society of America.

[42]  J. W. Gordon,et al.  Perceptual effects of spectral modifications on musical timbres , 1978 .

[43]  Nell P. McAngusTodd,et al.  The dynamics of dynamics: A model of musical expression , 1992 .

[44]  Giovanni De Poli,et al.  Fuzzy performance rules , 1995 .