Multidisciplinary Perspectives on Music Emotion Recognition: Implications for Content and Context-Based Models

The prominent status of music in human culture and every day life is due in large part to its striking ability to elicit emotions, which may manifest from slight variation in mood to changes in our physical condition and actions. In this paper, we first review state of the art stud- ies on music and emotions from dierent disciplines including psychology, musicology and music information retrieval. Based on these studies, we then propose new insights to enhance automated music emotion recog- nition models.

[1]  Tuomas Eerola,et al.  Measuring music-induced emotion , 2011 .

[2]  P. Young,et al.  Emotion and personality , 1963 .

[3]  Jeffrey J. Scott,et al.  State of the Art Report: Music Emotion Recognition: A State of the Art Review , 2010, ISMIR.

[4]  Tuomas Eerola,et al.  Generalizability and Simplicity as Criteria in Feature Selection: Application to Mood Classification in Music , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[5]  A. Gabrielsson,et al.  Emotional expression in synthesizer and sentograph performance. , 1995 .

[6]  Craig A. Smith,et al.  Appraisal theory: Overview, assumptions, varieties, controversies. , 2001 .

[7]  Piotr Synak,et al.  Multi-Label Classification of Emotions in Music , 2006, Intelligent Information Systems.

[8]  Sung Min Kim,et al.  Smoodi: Mood-based music recommendation player , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[9]  Emery Schubert.  Measuring emotion continuously: Validity and reliability of the two-dimensional emotion-space , 1999 .

[10]  Mark B. Sandler,et al.  The Music Ontology , 2007, ISMIR.

[11]  Francis F. Li,et al.  Music Mood Classification of Television Theme Tunes , 2011, ISMIR.

[12]  Tobias Brosch,et al.  Culture‐specific appraisal biases contribute to emotion dispositions , 2009 .

[13]  Kiyoaki Shirai,et al.  Machine Learning Approaches for Mood Classification of Songs toward Music Search Engine , 2009, 2009 International Conference on Knowledge and Systems Engineering.

[14]  Yi-Hsuan Yang,et al.  Music emotion classification: a fuzzy approach , 2006, MM '06.

[15]  C. Osgood,et al.  The Measurement of Meaning , 1958 .

[16]  Yi-Hsuan Yang,et al.  A Regression Approach to Music Emotion Recognition , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[17]  Jeffrey J. Scott,et al.  MUSIC EMOTION RECOGNITION: A STATE OF THE ART REVIEW , 2010 .

[18]  Trevor J. Cox,et al.  Musical Moods: A Mass Participation Experiment for Affective Classification of Music , 2011, ISMIR.

[19]  J. Stephen Downie,et al.  Survey Of Music Information Needs, Uses, And Seeking Behaviours: Preliminary Findings , 2004, ISMIR.

[20]  Gaël Richard,et al.  Musical instrument recognition by pairwise classification strategies , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[21]  K. Scherer,et al.  Emotions evoked by the sound of music: characterization, classification, and measurement. , 2008, Emotion.

[22]  Youngmoo E. Kim,et al.  Feature selection for content-based, time-varying musical emotion regression , 2010, MIR '10.

[23]  Tijl De Bie,et al.  Mining the Correlation between Lyrical and Audio Features and the Emergence of Mood , 2011, ISMIR.

[24]  Roddy Cowie,et al.  Tracing Emotion: An Overview , 2012, Int. J. Synth. Emot..

[25]  Cynthia Whissell,et al.  THE DICTIONARY OF AFFECT IN LANGUAGE , 1989 .

[26]  Ei Ei Pe Myint,et al.  An approach for mulit-label music mood classification , 2010, 2010 2nd International Conference on Signal Processing Systems.

[27]  Juan Pablo Bello,et al.  Automated Music Emotion Recognition: A Systematic Evaluation , 2010 .

[28]  Yi-Hsuan Yang,et al.  Prediction of the Distribution of Perceived Music Emotions Using Discrete Samples , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[29]  Klaus R. Scherer,et al.  Advocating a Componential Appraisal Model to Guide Emotion Recognition , 2012, Int. J. Synth. Emot..

[30]  C. Krumhansl.  An exploratory study of musical emotions and psychophysiology. , 1997, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[31]  Youngmoo E. Kim,et al.  Prediction of Time-varying Musical Mood Distributions from Audio , 2010, ISMIR.

[32]  Xing Wang,et al.  Music Emotion Classification of Chinese Songs based on Lyrics Using TF*IDF and Rhyme , 2011, ISMIR.

[33]  Petri Toiviainen,et al.  Prediction of Multidimensional Emotional Ratings in Music from Audio Using Multivariate Regression Models , 2009, ISMIR.

[34]  Jing Liu,et al.  The analysis of mood taxonomy comparision between chinese and western music , 2010, 2010 2nd International Conference on Signal Processing Systems.

[35]  Björn W. Schuller,et al.  Multi-Modal Non-Prototypical Music Mood Analysis in Continuous Space: Reliability and Performances , 2011, ISMIR.

[36]  Deshun Yang,et al.  Multi-Modal Music Mood Classification Using Co-Training , 2010, 2010 International Conference on Computational Intelligence and Software Engineering.

[37]  Gert R. G. Lanckriet,et al.  Modeling Dynamic Patterns for Emotional Content in Music , 2011, ISMIR.

[38]  Wolfgang Nejdl,et al.  Music Mood and Theme Classification - a Hybrid Approach , 2009, ISMIR.

[39]  Wolfgang Nejdl,et al.  Can all tags be used for search? , 2008, CIKM '08.

[40]  Seungmin Rho,et al.  SMERS: Music Emotion Recognition Using Support Vector Regression , 2009, ISMIR.

[41]  Youngmoo E. Kim,et al.  Prediction of Time-Varying Musical Mood Distributions Using Kalman Filtering , 2010, 2010 Ninth International Conference on Machine Learning and Applications.

[42]  T. Eerola,et al.  A comparison of the discrete and dimensional models of emotion in music , 2011 .

[43]  K. Scherer,et al.  Appraisal processes in emotion: Theory, methods, research. , 2001 .

[44]  J. Stephen Downie,et al.  Exploring Mood Metadata: Relationships with Genre, Artist and Usage Metadata , 2007, ISMIR.

[45]  Yi-Hsuan Yang,et al.  Ranking-Based Emotion Recognition for Music Organization and Retrieval , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[46]  John Z. Zhang,et al.  An Empirical Study of Multi-Label Classifiers for Music Tag Annotation , 2011, ISMIR.

[47]  Lie Lu,et al.  Automatic mood detection and tracking of music audio signals , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[48]  A. Gabrielsson,et al.  The influence of musical structure on emotional expression. , 2001 .

[49]  Leonard B. Meyer.  Emotion and Meaning in Music , 1957 .

[50]  K. Hevner.  Expression in music: a discussion of experimental studies and theories. , 1935 .

[51]  Youngmoo E. Kim,et al.  Modeling Musical Emotion Dynamics with Conditional Random Fields , 2011, ISMIR.

[52]  Gert R. G. Lanckriet,et al.  Combining audio content and social context for semantic music discovery , 2009, SIGIR.

[53]  Marc Leman,et al.  A User-Oriented Approach to Music Information Retrieval , 2006, Content-Based Retrieval.

[54]  Jun Wang,et al.  Enriching music mood annotation by semantic association reasoning , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[55]  Caridakis,et al.  Body gesture and facial expression analysis for automatic affect recognition , 2010 .

[56]  Seungjae Lee,et al.  Music mood classification model based on arousal-valence values , 2011, 13th International Conference on Advanced Communication Technology (ICACT2011).

[57]  Emiru Tsunoo,et al.  Music mood classification by rhythm and bass-line unit pattern analysis , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[58]  J. Russell.  A circumplex model of affect. , 1980 .

[59]  Yueting Zhuang,et al.  Popular music retrieval by detecting mood , 2003, SIGIR.

[60]  K. Scherer,et al.  Acoustic profiles in vocal emotion expression. , 1996, Journal of personality and social psychology.

[61]  Tao Li,et al.  Detecting emotion in music , 2003, ISMIR.

[62]  Yi-Hsuan Yang,et al.  Exploiting online music tags for music emotion classification , 2011, TOMCCAP.

[63]  Emery Schubert.  Update of the Hevner Adjective Checklist , 2003, Perceptual and motor skills.

[64]  K. Scherer,et al.  The World of Emotions is not Two-Dimensional , 2007, Psychological science.

[65]  K. Hevner.  Experimental studies of the elements of expression in music , 1936 .

[66]  Frederick kang'ethe Iraki,et al.  Presenting the VENEC corpus : Development of a cross-cultural corpus of vocal emotion expressions and a novel method of annotating emotion appraisals , 2010, LREC 2010.

[67]  Yi-Hsuan Yang,et al.  Exploiting genre for music emotion classification , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[68]  J. Sloboda,et al.  Psychological perspectives on music and emotion , 2001 .