Ai and Music: From Composition to Expressive Performance

In this article, we first survey the three major types of computer music systems based on AI techniques: (1) compositional, (2) improvisational, and (3) performance systems. Representative examples of each type are briefly described. Then, we look in more detail at the problem of endowing the resulting performances with the expressiveness that characterizes human-generated music. This is one of the most challenging aspects of computer music that has been addressed just recently. The main problem in modeling expressiveness is to grasp the performer's "touch," that is, the knowledge applied when performing a score. Humans acquire it through a long process of observation and imitation. For this reason, previous approaches, based on following musical rules trying to capture interpretation knowledge, had serious limitations. An alternative approach, much closer to the observation-imitation process observed in humans, is that of directly using the interpretation knowledge implicit in examples extracted from recordings of human performers instead of trying to make explicit such knowledge. In the last part of the article, we report on a performance system, SAXEX, based on this alternative approach, that is capable of generating high-quality expressive solo performances of jazz ballads based on examples of human performers within a case-based reasoning (CBR) system.

[1]  George Papadopoulos,et al.  A Genetic Algorithm for the Generation of Jazz Melodies , 2000 .

[2]  Roberto Bresin,et al.  Artificial neural networks based models for automatic performance of musical scores , 1998 .

[3]  Stephan M. Schwanauer.  A learning machine for tonal composition , 1992 .

[4]  Maarten Grachten.  JIG : Jazz Improvisation Generator , 2001 .

[5]  Roger A. Kendall,et al.  The Communication of Musical Expression , 1990 .

[6]  Gerhard Widmer,et al.  Learning expressive performance: The structure‐level approach , 1996 .

[7]  B. Baars.  A cognitive theory of consciousness , 1988 .

[8]  Ramon López de Mántaras,et al.  Combining fuzzy and case-based reasoning to generate human-like music performances , 2002 .

[9]  J. Sloboda.  Music Structure and Emotional Response: Some Empirical Findings , 1991 .

[10]  P. Johnson-Laird.  Jazz Improvization: A Theory at the Computational Level , 1991 .

[11]  Janet L. Kolodner,et al.  Case-Based Reasoning , 1989, IJCAI 1989.

[12]  Ray Jackendoff,et al.  An overview of hierarchical structure in music , 1983 .

[13]  M. Thaut,et al.  The Influence of Preferred Relaxing Music on Measures of State Anxiety, Relaxation, and Physiological Responses , 1989 .

[14]  C. Krumhansl.  Music as Cognition. , 1987 .

[15]  J. Sloboda.  Generative Processes in Music: The Psychology of Performance, Improvisation, and Composition , 1985 .

[16]  Josep Lluis Arcos,et al.  The Noos Representation Language , 1997 .

[17]  M. Minsky.  Music, Mind, and Meaning , 1981 .

[18]  Agnar Aamodt,et al.  Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches , 1994, AI Commun..

[19]  Roger B. Dannenberg,et al.  Combining Instrument and Performance Models for High-Quality Music Synthesis , 1998 .

[20]  Johan Sundberg,et al.  Music from Motion: Sound Level Envelopes of Tones Expressing Human Locomotion , 2000 .

[21]  John A. Biles,et al.  GenJam: A Genetic Algorithm for Generating Jazz Solos , 1994, ICMC.

[22]  Gerhard Widmer,et al.  The Musical Expression Project: A Challenge for Machine Learning and Knowledge Discovery , 2001, PKDD.

[23]  John Rothgeb.  Simulating musical skills by digital computer , 1979, ACM '79.

[24]  Leonard Isaacson,et al.  Musical composition with a high-speed digital computer , 1957 .

[25]  Kemal Ebcioglu,et al.  An Expert System for Harmonizing Four-Part Chorales , 1988, ICMC.

[26]  François Pachet,et al.  Formulating Constraint Satisfaction Problems on Part-Whole Relations : The Case of Automatic Musical Harmonization , 1998 .

[27]  J. Sabater-Mir,et al.  Using Rules to support Case-Based Reasoning for harmonizing melodies , 1998, AAAI 1998.

[28]  David Cope.  Pattern Matching as an Engine for the Computer Simulation of Musical Style , 1990, ICMC.

[29]  Ramón López de Mántaras,et al.  An Interactive Case-Based Reasoning Approach for Generating Expressive Music , 2004, Applied Intelligence.

[30]  David Leake,et al.  Case-Based Reasoning: Experiences, Lessons and Future Directions , 1996 .

[31]  Eugene Narmour,et al.  The Analysis and Cognition of Basic Melodic Structures: The Implication-Realization Model , 1990 .

[32]  Johan Sundberg,et al.  Musical punctuation on the microlevel : Automatic identification and performance of small melodic units , 1998 .

[33]  Johannes Feulner,et al.  Neural Networks that Learn and Reproduce Various Styles of Harmonization , 1993, ICMC.

[34]  James Anderson Moorer.  Music and computer composition , 1972, CACM.

[35]  Roger B. Dannenberg,et al.  Software support for interactive multimedia performance , 1993 .

[36]  Jeff Pressing,et al.  Improvisation: Methods and models. , 1988 .

[37]  Judy A. Franklin.  Multi-Phase Learning for Jazz Improvisation and Interaction , 2001 .

[38]  Roberto Bresin,et al.  Articulation Rules For Automatic Music Performance , 2001, ICMC.

[39]  Wolfram Menzel,et al.  Learning musical structure and style with neural networks , 1998 .

[40]  Matthew Wright,et al.  Preparation for Improvised Performance in Collaboration with a Khyal Singer , 1998, ICMC.

[41]  Dominik Hörnel,et al.  A Neural Organist improvising baroque-style melodic variations , 1997, ICMC.

[42]  Stephan M. Schwanauer,et al.  Machine Models of Music , 1993 .

[43]  C. Robazza,et al.  Emotional Reactions to Music by Gender, Age, and Expertise , 1994, Perceptual and motor skills.

[44]  Xavier Serra,et al.  SaxEx: a case-based reasoning system for generating expressive musical performances , 1998, ICMC.

[45]  Gary M. Rader,et al.  A method for composing simple traditional music by computer , 1974, CACM.

[46]  Roger B. Dannenberg,et al.  SICIB: An Interactive Music Composition System Using Body Movements , 2001, Computer Music Journal.

[47]  C. Krumhansl.  An exploratory study of musical emotions and psychophysiology. , 1997, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[48]  Jamshed J. Bharucha,et al.  MUSACT: a connectionist model of musical harmony , 1992 .

[49]  Manuela Veloso,et al.  Bob: an improvisational music companion , 2001 .

[50]  William A. Woods,et al.  Computational Linguistics Transition Network Grammars for Natural Language Analysis , 2022 .

[51]  David Levitt,et al.  A representation for musical dialects , 1992 .

[52]  Anders Friberg,et al.  A Quantitative Rule System for Musical Performance , 1995 .

[53]  Alan C. Evans,et al.  Emotional responses to pleasant and unpleasant music correlate with activity in paralimbic brain regions , 1999, Nature Neuroscience.

[54]  A. Goldstein.  Thrills in response to music and other stimuli , 1980 .

[55]  Christopher Fry,et al.  Flavors Band: a language for specifying musical style , 1984 .

[56]  Xavier Serra,et al.  Integrating complementary spectral models in the design of a musical synthesizer , 1997, ICMC.