The Perception of Emotion in the Singing Voice: The Understanding of Music Mood for Music Organisation

With the increased usage of internet based services and the mass of digital content now available online, the organisation of such content has become a major topic of interest both commercially and within academic research. The addition of emotional understanding for the content is a relevant parameter not only for music classification within digital libraries but also for improving users experiences, via services including automated music recommendation. Despite the singing voice being well-known for the natural communication of emotion, it is still unclear which specific musical characteristics of this signal are involved such affective expressions. The presented study investigates which musical parameters of singing relate to the emotional content, by evaluating the perception of emotion in electronically manipulated a cappella audio samples. A group of 24 individuals participated in a perception test evaluating the emotional dimensions of arousal and valence of 104 sung instances. Key results presented indicate that the rhythmic-melodic contour is potentially related to the perception of arousal whereas musical syntax and tempo can alter the perception of valence.

[1]  Gail M. Sullivan,et al.  Using Effect Size-or Why the P Value Is Not Enough. , 2012, Journal of graduate medical education.

[2]  Gerrit Bloothooft,et al.  Perception and Acoustics of Emotions in Singing , 1997, EUROSPEECH.

[3]  Bruno Nettl,et al.  Thoughts on Improvisation: A Comparative Approach , 1974 .

[4]  Akinori Ito,et al.  A System for Evaluating Singing Enthusiasm for Karaoke , 2011, ISMIR.

[5]  Klaus R. Scherer,et al.  Vocal communication of emotion , 2000 .

[6]  D. Heylen,et al.  Issues in Data Labelling , 2011 .

[7]  György Fazekas,et al.  A Study of Cultural Dependence of Perceived Mood in Greek Music , 2013, ISMIR.

[8]  Roddy Cowie,et al.  FEELTRACE: an instrument for recording perceived emotion in real time , 2000 .

[9]  Emery Schubert.  Measuring emotion continuously: Validity and reliability of the two-dimensional emotion-space , 1999 .

[10]  Lori Lamel,et al.  Challenges in real-life emotion annotation and machine learning based detection , 2005, Neural Networks.

[11]  Björn W. Schuller,et al.  Determination of Nonprototypical Valence and Arousal in Popular Music: Features and Performances , 2010, EURASIP J. Audio Speech Music. Process..

[12]  Frank A. Russo,et al.  Acoustic differences in the speaking and singing voice , 2013 .

[13]  J. Stephen Downie,et al.  Exploring Mood Metadata: Relationships with Genre, Artist and Usage Metadata , 2007, ISMIR.

[14]  Xavier Serra.  A Multicultural Approach in Music Information Research , 2011, ISMIR.

[15]  Xiao Hu,et al.  Towards global music digital libraries: A cross-cultural comparison on the mood of Chinese music , 2016, J. Documentation.

[16]  Eliezer Rapoport.  Emotional expression code in opera and lied singing , 1996 .

[17]  P. Laukka,et al.  Expression, Perception, and Induction of Musical Emotions: A Review and a Questionnaire Study of Everyday Listening , 2004 .

[18]  Kim E. A. Silverman,et al.  Vocal cues to speaker affect: testing two models , 1984 .

[19]  Mert Bay,et al.  The 2007 MIREX Audio Mood Classification Task: Lessons Learned , 2008, ISMIR.

[20]  A. Fischer.  Sex Differences in Emotionality: Fact or Stereotype? , 1993 .

[21]  Giovanni Comotti,et al.  Music in Greek and Roman culture , 1989 .

[22]  Björn W. Schuller,et al.  Multi-Modal Non-Prototypical Music Mood Analysis in Continuous Space: Reliability and Performances , 2011, ISMIR.

[23]  Kisang Ryu,et al.  The Effect of Environmental Perceptions on Behavioral Intentions Through Emotions: The Case of Upscale Restaurants , 2007 .

[24]  W. Thompson,et al.  A Comparison of Acoustic Cues in Music and Speech for Three Dimensions of Affect , 2006 .

[25]  S. McAdams,et al.  Music induces universal emotion-related psychophysiological responses: comparing Canadian listeners to Congolese Pygmies , 2015, Front. Psychol..

[26]  Jin Ha Lee,et al.  What does music mood mean for real users? , 2012, iConference '12.

[27]  Catherine J. Stevens,et al.  Music Perception and Cognition: A Review of Recent Cross-Cultural Research , 2012, Top. Cogn. Sci..

[28]  Vladimir P. Morozov.  Emotional expressiveness of the singing voice: The role of macrostructural and microstructural modifications of spectra , 1996 .

[29]  K. Scherer,et al.  Acoustic concomitants of emotional expression in operatic singing: the case of Lucia in Ardi gli incensi. , 1995, Journal of voice : official journal of the Voice Foundation.

[30]  Bruno H. Repp.  Quantitative Effects of Global Tempo on Expressive Timing in Music Performance: Some Perceptual Evidence , 1995 .

[31]  W. Thompson,et al.  Recognition of emotion in Japanese, Western, and Hindustani music by Japanese listeners 1 , 2004 .

[32]  Björn W. Schuller,et al.  Emotion in the singing voice—a deeperlook at acoustic features in the light ofautomatic classification , 2015, EURASIP J. Audio Speech Music. Process..

[33]  Daniel G. Brown,et al.  On Cultural, Textual and Experiential Aspects of Music Mood , 2014, ISMIR.

[34]  Johan Sundberg,et al.  A singer ' s expression of emotions in sung performance , 2007 .

[35]  K. Scherer,et al.  Emotions evoked by the sound of music: characterization, classification, and measurement. , 2008, Emotion.

[36]  Iain R. Murray,et al.  Toward the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion. , 1993, The Journal of the Acoustical Society of America.

[37]  David Buxton,et al.  Rock Music, the Star-System and the Rise of Consumerism , 1983, Telos.

[38]  Jakob Wisse,et al.  Cicero: On the Ideal Orator (De Oratore) , 2001 .

[39]  Fabien Ringeval,et al.  Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions , 2013, 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[40]  K. Scherer,et al.  Vocal cues to deception: A comparative channel approach , 1985, Journal of psycholinguistic research.

[41]  H Fukui,et al.  Music and testosterone. A new hypothesis for the origin and function of music. , 2001, Annals of the New York Academy of Sciences.

[42]  G Anna Piotrowska,et al.  The place of ‘Russian music’ on the multicultural map of Europe , 2016 .

[43]  N. Lazar,et al.  The ASA Statement on p-Values: Context, Process, and Purpose , 2016 .

[44]  Klaus R. Scherer,et al.  Vocal communication of emotion: A review of research paradigms , 2003, Speech Commun..

[45]  Habib Hassan Touma,et al.  The Maqam Phenomenon: An Improvisation Technique in the Music of the Middle East , 1971 .

[46]  Johan Sundberg,et al.  Comparing the acoustic expression of emotion in the speaking and the singing voice , 2015, Comput. Speech Lang..

[47]  K. Scherer,et al.  On the Acoustics of Emotion in Audio: What Speech, Music, and Sound have in Common , 2013, Front. Psychol..

[48]  Roddy Cowie,et al.  Beyond emotion archetypes: Databases for emotion modelling using neural networks , 2005, Neural Networks.

[49]  P. Laukka,et al.  Communication of emotions in vocal expression and music performance: different channels, same code? , 2003, Psychological bulletin.

[50]  Clifton Ware,et al.  Basics of vocal pedagogy : the foundations and process of singing , 1997 .

[51]  J. Russell.  A circumplex model of affect. , 1980 .

[52]  Björn W. Schuller,et al.  iHEARu-PLAY: Introducing a game for crowdsourced data collection for affective computing , 2015, 2015 International Conference on Affective Computing and Intelligent Interaction (ACII).

[53]  Björn W. Schuller,et al.  On-line continuous-time music mood regression with deep recurrent neural networks , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[54]  P. Ekman.  Expression and the Nature of Emotion , 1984 .

[55]  P. Bohlman,et al.  The Study of Folk Music in the Modern World , 2021 .

[56]  John Shepherd,et al.  The Routledge reader on the sociology of music , 2015 .

[57]  Eduardo Coutinho,et al.  Transfer learning emotion manifestation across music and speech , 2014, 2014 International Joint Conference on Neural Networks (IJCNN).