Automated Analysis of Body Movement in Emotionally Expressive Piano Performances

EMOTIONAL EXPRESSION IN MUSIC PERFORMANCE includes important cues arising from the body movement of the musician. This movement is related to both the musical score execution and the emotional intention conveyed. In this experiment, a pianist was asked to play the same excerpt with different emotionally expressive intentions. The aim was to verify whether different expressions could be distinguished based on movement by trying to determine which motion cues were most emotion-sensitive. Analyses were performed via an automated system capable of detecting the temporal profiles of two motion cues: the quantity of motion of the upper body and the velocity of head movements. Results showed that both were sensitive to emotional expression, especially the velocity of head movements. Further, some features conveying information about movement temporal dynamics varied among expressive conditions allowing emotion discrimination. These results are in line with recent theories that underlie the dynamic nature of emotional expression.

[1]  DeWitt H. Parker,et al.  Philosophy in a New Key , 1941 .

[2]  Rudolf von Laban,et al.  Modern Educational Dance , 1948 .

[3]  P. Ekman,et al.  Detecting deception from the body or face. , 1974 .

[4]  K. Scherer.  Personality inference from voice quality: The loud voice of extroversion. , 1978 .

[5]  K. Scherer,et al.  Cues and channels in emotion recognition. , 1986 .

[6]  E. Clarke.  Generative principles in music performance. , 1988 .

[7]  Johan Sundberg,et al.  Rules for Expression in the Performance of Melodies , 1989 .

[8]  M. D. Meijer.  The contribution of general features of body movement to the attribution of emotions , 1989 .

[9]  Johan Sundberg,et al.  Common Secrets of Musicians and Listeners - An analysis-by-synthesis Study of Musical Performance , 1991 .

[10]  P. Juslin.  Emotional communication in music viewed through a Brunswikian lens , 1995 .

[11]  P. Juslin,et al.  Emotional Expression in Music Performance: Between the Performer's Intention and the Listener's Experience , 1996 .

[12]  P. Juslin.  Emotional Communication in Music Performance: A Functionalist Perspective and Some Data , 1997 .

[13]  Patrik N. Juslin,et al.  Can results from studies of perceived expression in musical performances be generalized across response formats , 1997 .

[14]  Giovanni De Poli,et al.  Note‐by‐note analysis of the influence of expressive intentions and musical structure in violin performance* , 1998 .

[15]  J. Cunningham,et al.  Children's decoding of emotion in expressive body movement: the development of cue attunement. , 1998, Developmental psychology.

[16]  H. Wallbott.  Bodily expression of emotion , 1998 .

[17]  Carlo Drioli,et al.  Audio Morphing Different Expressive Intentions for Multimedia Systems , 2000, IEEE Multim..

[18]  P. Juslin,et al.  Cue Utilization in Communication of Emotion in Music Performance: Relating Performance to Perception Studies of Music Performance , 2022 .

[19]  Armin Bruderlin,et al.  Perceiving affect from arm movement , 2001, Cognition.

[20]  K. Scherer.  Appraisal considered as a process of multilevel sequential checking. , 2001 .

[21]  K. Scherer,et al.  EMOTIONAL EFFECTS OF MUSIC: PRODUCTION RULES , 2001 .

[22]  Emery Schubert.  Correlation analysis of continuous emotional response to music: Correcting for the effects of serial correlation , 2001 .

[23]  J. Sloboda,et al.  Psychological perspectives on music and emotion , 2001 .

[24]  A. Gabrielsson,et al.  The influence of musical structure on emotional expression. , 2001 .

[25]  Antonio Camurri,et al.  Multimodal Analysis of Expressive Gesture in Music and Dance Performances , 2003, Gesture Workshop.

[26]  Marcelo M. Wanderley,et al.  Performance Gestures of Musicians: What Structural and Emotional Information Do They Convey? , 2003, Gesture Workshop.

[27]  Antonio Camurri,et al.  Analysis of Expressive Gesture: The EyesWeb Expressive Gesture Processing Library , 2003, Gesture Workshop.

[28]  Sethuraman Panchanathan,et al.  Gesture segmentation in complex motion sequences , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[29]  Antonio Camurri,et al.  Recognizing emotion from dance movement: comparison of spectator recognition and automated techniques , 2003, Int. J. Hum. Comput. Stud..

[30]  P. Juslin,et al.  Emotional expression in music. , 2003 .

[31]  Klaus R. Scherer,et al.  Vocal communication of emotion: A review of research paradigms , 2003, Speech Commun..

[32]  Facial expressions of emotion in speech and singing , 2004 .

[33]  K. Scherer.  Which Emotions Can be Induced by Music? What Are the Underlying Mechanisms? And How Can We Measure Them? , 2004 .

[34]  P. Laukka,et al.  Expression, Perception, and Induction of Musical Emotions: A Review and a Questionnaire Study of Everyday Listening , 2004 .

[35]  Kostas Karpouzis,et al.  Emotion Analysis in Man-Machine Interaction Systems , 2004, MLMI.

[36]  Frank A. Russo,et al.  Seeing music performance: Visual influences on perception and experience , 2005 .

[37]  Daniel J. Levitin,et al.  Analyzing Temporal Dynamics in Music: : Differential Calculus, Physics, and Functional Data Analysis Techniques , 2005 .

[38]  G. Luck,et al.  Ensemble Musicians’ Synchronization With Conductors’ Gestures: An Automated Feature-Extraction Analysis , 2006 .

[39]  William Forde Thompson,et al.  MULTIMODAL AFFECTIVE INTERACTION: A COMMENT ON MUSICAL ORIGINS , 2006 .

[40]  Antonio Camurri,et al.  Listeners' emotional engagement with performances of a Scriabin étude: an explorative case study , 2006 .

[41]  W. Thompson,et al.  Multimodal Affective Interaction , 2006 .

[42]  Hatice Gunes,et al.  Bi-modal emotion recognition from expressive face and body gestures , 2007, J. Netw. Comput. Appl..

[43]  Antonio Camurri,et al.  Developing multimodal interactive systems with EyesWeb XMI , 2007, NIME '07.

[44]  Peter Robinson,et al.  Detecting Affect from Non-stylised Body Motions , 2007, ACII.

[45]  Ginevra Castellano,et al.  Recognising Human Emotions from Body Movement and Gesture Dynamics , 2007, ACII.

[46]  A. Friberg,et al.  Visual Perception of Expressiveness in Musicians' Body Movements , 2007 .

[47]  Frank A. Russo,et al.  Audio-visual integration of emotional cues in song , 2008 .