Composing by Listening: A Computer-Assisted System for Creating Emotional Music

Most people communicate emotion through their voice, facial expressions, and gestures. However, it is assumed that only "experts" can communicate emotions in music. The authors have developed a computer-based system that enables musically untrained users to select relevant acoustic attributes to compose emotional melodies. Nonmusicians Experiment 1 and musicians Experiment 3 were progressively presented with pairs of melodies that each differed in an acoustic attribute e.g., intensity-loud vs. soft. For each pair, participants chose the melody that most strongly conveyed a target emotion anger, fear, happiness, sadness or tenderness. Once all decisions were made, a final melody containing all choices was generated. The system allowed both untrained and trained participants to compose a range of emotional melodies. New listeners successfully decoded the emotional melodies of nonmusicians Experiment 2 and musicians Experiment 4. Results indicate that human-computer interaction can facilitate the composition of emotional music by musically untrained and trained individuals.

[1]  C. Davies.  Listen to my song: a study of songs invented by children aged 5 to 7 years , 1992, British Journal of Music Education.

[2]  W. Thompson,et al.  A Cross-Cultural Investigation of the Perception of Emotion in Music: Psychophysical and Cultural Cues , 1999 .

[3]  Martin Fautley.  A new model of the group composing process of lower secondary school students , 2005 .

[4]  C. Seashore.  Measurements on the Expression of Emotion in Music. , 1923, Proceedings of the National Academy of Sciences of the United States of America.

[5]  W. Thompson,et al.  Can Composers Express Emotions through Music? , 1992 .

[6]  D. Campbell.  Blind variation and selective retention in creative thought as in other knowledge processes. , 1960, Psychological review.

[7]  Edward Iglesias.  Robots in Academic Libraries: Advancements in Library Automation , 2013 .

[8]  M. Terwogt,et al.  Musical Expression of Moodstates , 1991 .

[9]  C. Krumhansl.  Cognitive Foundations of Musical Pitch , 1990 .

[10]  D. Simonton.  REJOINDER: Picasso's Guernica Creativity as a Darwinian Process: Definitions, Clarifications, Misconceptions, and Applications , 2007 .

[11]  R. G. Crowder.  Perception of the major/minor distinction: I. Historical and theoretical foundations. , 1984 .

[12]  Andreas C. Lehmann,et al.  The Cambridge Handbook of Expertise and Expert Performance: Music , 2006 .

[13]  P. Juslin,et al.  Cue Utilization in Communication of Emotion in Music Performance: Relating Performance to Perception Studies of Music Performance , 2022 .

[14]  Kate Hevner,et al.  The affective character of the major and minor mode in music , 1935 .

[15]  Sergej Fatikow,et al.  Nanorobot-Based Handling and Transfer of Individual Silicon Nanowires , 2012, Int. J. Intell. Mechatronics Robotics.

[16]  P. Juslin,et al.  Expression and communication of emotion in music performance. , 2010 .

[17]  Patrik N. Juslin,et al.  Musical expression of emotions : Modeling listeners´ judgements of composed and performed features , 2010 .

[18]  J. Russell,et al.  Concept of Emotion Viewed From a Prototype Perspective , 1984 .

[19]  E. Bigand,et al.  Are we “experienced listeners”? A review of the musical capacities that do not depend on formal musical training , 2006, Cognition.

[20]  P. Laukka,et al.  Communication of emotions in vocal expression and music performance: different channels, same code? , 2003, Psychological bulletin.

[21]  B. Tillmann.  Implicit Investigations of Tonal Knowledge in Nonmusician Listeners , 2005, Annals of the New York Academy of Sciences.

[22]  A. Gabrielsson,et al.  The role of structure in the musical expression of emotions , 2010 .

[23]  P. Juslin,et al.  Emotional Expression in Music Performance: Between the Performer's Intention and the Listener's Experience , 1996 .

[24]  D. Simonton.  Creativity as Blind Variation and Selective Retention : Is the Creative Process Darwinian ? , 2022 .

[25]  A. Clark,et al.  The Extended Mind , 1998, Analysis.

[26]  M. P. Kastner.  Perception of major/minor : IV. Emotional connotations in young children , 1990 .

[27]  I. Peretz,et al.  A developmental study of the affective value of tempo and mode in music , 2001, Cognition.

[28]  J. Russell.  A circumplex model of affect. , 1980 .

[29]  D. Simonton.  Creativity and Discovery as Blind Variation: Campbell's (1960) BVSR Model after the Half-Century Mark , 2011 .

[30]  C. Robazza,et al.  Emotional Reactions to Music by Gender, Age, and Expertise , 1994, Perceptual and motor skills.

[31]  I. Peretz,et al.  Universal Recognition of Three Basic Emotions in Music , 2009, Current Biology.

[32]  Paul J. Feltovich,et al.  The Cambridge handbook of expertise and expert performance , 2006 .

[33]  Kristin Precoda,et al.  Influence of Pitch Height on the Perception of Submissiveness and Threat in Musical Passages , 2006 .

[34]  K. Scherer,et al.  Cue utilization in emotion attribution from auditory stimuli , 1977 .

[35]  Betty Anne Younker,et al.  Comparing and Modeling Musical Thought Processes of Expert and Novice Composers , 1996 .

[36]  Gary Cziko,et al.  Without Miracles: Universal Selection Theory and the Second Darwinian Revolution , 1995 .

[37]  Tim Cain,et al.  Theory, technology and the music curriculum , 2004, British Journal of Music Education.

[38]  Kari Kallinen,et al.  Emotional ratings of music excerpts in the western art music repertoire and their self-organization in the Kohonen neural network , 2005 .

[39]  W. Thompson,et al.  The contributions of compositional structure and performance expression to the communication of emotion in music , 2014 .

[40]  Peter Johnson,et al.  An Exploration of Constraints and End User Development in Environments for Creative Tasks , 2008, Int. J. Hum. Comput. Interact..

[41]  R. G. Crowder,et al.  Perception of the Major/Minor Distinction: IV. Emotional Connotations in Young Children , 1990 .

[42]  Roger A. Kendall,et al.  The Communication of Musical Expression , 1990 .

[43]  P. Juslin,et al.  The Role of Timing Patterns in Recognition of Emotional Expression from Musical Performance , 1999 .

[44]  Janet Mills,et al.  Music technology inspected: good teaching in Key Stage 3 , 2000, British Journal of Music Education.

[45]  Edward Iglesias.  The Inevitability of Library Automation , 2013 .

[46]  P. Herrero,et al.  A Cognitive Appraisal Based Approach for Emotional Representation , 2009 .