Affective communication remapping in MusicFace System

This paper addresses the issue of affective communication remapping, i.e. translation of affective content from one communication form to another. We propose a method to extract the affective data from a piece of music and then use that to animate a face. The method is based on studies of emotional aspect of music and our behavioural head model for face animation.

[1]  C. Darwin.  The Expression of the Emotions in Man and Animals , .

[2]  K. Hevner.  Experimental studies of the elements of expression in music , 1936 .

[3]  Charles J. Burkhart,et al.  The Social Psychology of Music , 1958 .

[4]  Norman I. Badler,et al.  Movies from music: Visualizing musical compositions , 1979, SIGGRAPH '79.

[5]  J. Russell.  A circumplex model of affect. , 1980 .

[6]  R. Thayer.  The biopsychology of mood and arousal , 1989 .

[7]  Manfred Clynes.  Time-forms, nature's generators and communicators of emotion , 1992, [1992] Proceedings IEEE International Workshop on Robot and Human Communication.

[8]  Adrian C. North,et al.  The social psychology of music. , 1997 .

[9]  Jörn Ostermann,et al.  Animation of synthetic faces in MPEG-4 , 1998, Proceedings Computer Animation '98 (Cat. No.98EX169).

[10]  Anders Friberg,et al.  Synthesis and decoding of emotionally expressive music performance , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[11]  P. Juslin,et al.  Cue Utilization in Communication of Emotion in Music Performance: Relating Performance to Perception Studies of Music Performance , 2022 .

[12]  Tim Valentine,et al.  Face–space models of face recognition. , 2001 .

[13]  J. Sloboda,et al.  Music and emotion: Theory and research , 2001 .

[14]  Steve DiPaola,et al.  FaceSpace: a facial spatial-domain toolkit , 2002, Proceedings Sixth International Conference on Information Visualisation.

[15]  C. Krumhansl.  Music: A Link Between Cognition and Emotion , 2002 .

[16]  Peter Robinson,et al.  Music-driven motion editing , 2002, SIGGRAPH '02.

[17]  Sung Yong Shin,et al.  Rhythmic-motion synthesis based on motion-beat analysis , 2003, ACM Trans. Graph..

[18]  Lie Lu,et al.  Automatic mood detection from acoustic music data , 2003, ISMIR.

[19]  Tao Li,et al.  Detecting emotion in music , 2003, ISMIR.

[20]  Justine Cassell,et al.  BEAT: the Behavior Expression Animation Toolkit , 2001, Life-like characters.

[21]  P. Ekman.  Emotions revealed , 2004, BMJ.

[22]  Robert O. Gjerdingen,et al.  The Cognition of Basic Musical Structures , 2004 .

[23]  Steve DiPaola,et al.  Face as multimedia object , 2004 .

[24]  Manfred Clynes.  Nature's Generators and Communicators of Emotion , 2005 .

[25]  J. Townsend,et al.  Computational, Geometric, and Process Perspectives on Facial Cognition : Contexts and Challenges , 2005 .