Automatic Classification of musical mood by content-based analysis

La musica en formato digital forma parte de nuestras vidas. Automatizar la organizacion de estos datos es un gran desafio. En esta tesis, nos centramos en la clasificacion automatica de musica a partir de la deteccion de la emocion que comunica. Para conseguirlo, proponemos modelos usando informaciones extraidas de la senal de audio mediante tecnicas de procesamiento de senales, aprendizaje automatico y recuperacion de informacion. Primero, estudiamos como los miembros de una red social utilizan etiquetas y palabras clave para describir la musica y las emociones que evoca. Con una tecnica para reducir la complejidad dimensional de este problema, encontramos un modelo para representar los estados de animo. Luego, proponemos un metodo de clasificacion automatica de emociones y detallamos los resultados para distintos tipos de clasificadores. Analizamos las contribuciones de descriptores de audio y como sus valores estan relacionados con los estados de animo, intentando encontrar explicaciones desde un punto de vista psicologico y/o musicologico. Proponemos tambien una version multimodal de nuestro algoritmo, usando las letras de canciones con un nuevo metodo de clasificacion basado en las palabras claves para distinguir categorias de emociones. Finalmente, despues de estudiar la relacion entre el estado de animo y el genero musical, presentamos un metodo usando la clasificacion automatica por genero. Mostramos que clasificadores basados en el genero obtienen mejores resultados que otros metodos estandar. A modo de recapitulacion conceptual y algoritmica, proponemos una tecnica de extraccion de reglas para entender como los algoritmos de aprendizaje automatico predicen la emocion evocada por la musica. Nuestros algoritmos han sido evaluados con datos de usuarios y en concursos de evaluacion internacionales.

[1]  J. Suykens,et al.  Benchmarking state-of-the-art classification algorithms for credit scoring: An update of research , 2015, Eur. J. Oper. Res..

[2]  Xavier Serra,et al.  Unifying Low-Level and High-Level Music Similarity Measures , 2011, IEEE Transactions on Multimedia.

[3]  S. Groux Situated, perceptual, emotive and cognitive music systems: a psychologically grounded approach to interactive music composition , 2011 .

[4]  Yi-Hsuan Yang,et al.  Ranking-Based Emotion Recognition for Music Organization and Retrieval , 2011, IEEE Transactions on Audio, Speech, and Language Processing.

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  T. Eerola,et al.  A comparison of the discrete and dimensional models of emotion in music , 2011 .

[7]  J. Stephen Downie,et al.  When Lyrics Outperform Audio for Music Mood Classification: A Feature Analysis , 2010, ISMIR.

[8]  Bipin Indurkhya,et al.  A behavioral study of emotions in south indian classical music andits implications in music recommendation systems , 2010, SAPMIA '10.

[9]  J. Stephen Downie,et al.  Improving mood classification in music digital libraries by combining lyrics and audio , 2010, JCDL '10.

[10]  Xavier Serra,et al.  Indexing music by mood: design and integration of an automatic content-based annotator , 2010, Multimedia Tools and Applications.

[11]  Youngmoo E. Kim,et al.  Feature selection for content-based, time-varying musical emotion regression , 2010, MIR '10.

[12]  Andreas F. Ehmann,et al.  Lyric Text Mining in Music Mood Classification , 2009, ISMIR.

[13]  Wolfgang Nejdl,et al.  Pharos: an audiovisual search platform , 2009, SIGIR.

[14]  Yi-Hsuan Yang,et al.  Exploiting genre for music emotion classification , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[15]  Suh-Yin Lee,et al.  Emotion-based music recommendation by affinity discovery from film music , 2009, Expert Syst. Appl..

[16]  I. Peretz,et al.  Universal Recognition of Three Basic Emotions in Music , 2009, Current Biology.

[17]  Bart Baesens,et al.  Decompositional Rule Extraction from Support Vector Machines by Active Learning , 2009, IEEE Transactions on Knowledge and Data Engineering.

[18]  Hui He,et al.  Language Feature Mining for Music Emotion Classification via Supervised Learning from Lyrics , 2008, ISICA.

[19]  Jens Grivolla,et al.  Multimodal Music Mood Classification Using Audio and Lyrics , 2008, 2008 Seventh International Conference on Machine Learning and Applications.

[20]  Yi-Hsuan Yang,et al.  Toward Multi-modal Music Emotion Classification , 2008, PCM.

[21]  Stefanie Nowak,et al.  Content-based mood classification for photos and music: a generic multi-modal classification framework and evaluation approach , 2008, MIR '08.

[22]  Andreas Rauber,et al.  Combination of audio and lyrics features for genre classification in digital audio collections , 2008, ACM Multimedia.

[23]  G. Peeters,et al.  A Generic Training and Classification System for MIREX08 Classification Tasks: Audio Music Mood, Audio Genre, Audio Artist and Audio Tag , 2008 .

[24]  Emilia Gómez,et al.  Comparative Analysis of Music Recordings from Western and Non-Western traditions by Automatic Tonal Feature Extraction , 2008 .

[25]  W. Dowling Emotion and Meaning in Music , 2008 .

[26]  I. Peretz,et al.  Happy, sad, scary and peaceful musical excerpts for research on emotions , 2008 .

[27]  Yi-Hsuan Yang,et al.  A Regression Approach to Music Emotion Recognition , 2008, IEEE Transactions on Audio, Speech, and Language Processing.

[28]  Petri Toiviainen,et al.  Modelling the relationships between emotional responses to, and musical content of, music therapy improvisations , 2008 .

[29]  Aniruddh D. Patel Music, Language, and the Brain , 2007 .

[30]  J. Stephen Downie,et al.  Exploring Mood Metadata: Relationships with Genre, Artist and Usage Metadata , 2007, ISMIR.

[31]  E. Altenmüller,et al.  Emotions over time: synchronicity and development of subjective, physiological, and facial affective reactions to music. , 2007, Emotion.

[32]  C. Stevens,et al.  Sweet Anticipation: Music and the Psychology of Expectation, by David Huron . Cambridge, Massachusetts: MIT Press, 2006 , 2007 .

[33]  Yannis Stylianou,et al.  A Statistical Approach to Musical Genre Classification using Non-Negative Matrix Factorization , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[34]  Andreas Rauber,et al.  Integration of Text and Audio Features for Genre Classification in Music Information Retrieval , 2007, ECIR.

[35]  Carol L. Krumhansl,et al.  Modeling Tonal Tension , 2007 .

[36]  Lie Lu,et al.  Automatic mood detection and tracking of music audio signals , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[37]  Kong Joo Lee,et al.  Automatic Affect Recognition Using Natural Language Processing Techniques and Manually Built Affect Lexicon , 2006, IEICE Trans. Inf. Syst..

[38]  Yi-Hsuan Yang,et al.  Music emotion classification: a fuzzy approach , 2006, MM '06.

[39]  Stefan Koelsch,et al.  The Role of Harmonic Expectancy Violations in Musical Emotions: Evidence from Subjective, Physiological, and Neural Responses , 2006, Journal of Cognitive Neuroscience.

[40]  Daniel P. W. Ellis,et al.  Support vector machine active learning for music retrieval , 2006, Multimedia Systems.

[41]  Emilia Gómez Gutiérrez,et al.  Tonal description of music audio signals , 2006 .

[42]  Xuan Zhu,et al.  A Tempo Feature via Modulation Spectrum Analysis and its Application to Music Emotion Classification , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[43]  A. Friederici,et al.  Investigating emotion with music: An fMRI study , 2006, Human brain mapping.

[44]  J. Marozeau,et al.  Multidimensional scaling of emotional responses to music: The effect of musical expertise and of the duration of the excerpts , 2005 .

[45]  Markus Koppenberger,et al.  Natural language processing of lyrics , 2005, ACM Multimedia.

[46]  M. Hubert,et al.  Robust classification in high dimensions based on the SIMCA Method , 2005 .

[47]  Cecilia Ovesdotter Alm,et al.  Emotions from Text: Machine Learning for Text-based Emotion Prediction , 2005, HLT.

[48]  Daniel J. Levitin,et al.  The rewards of music listening: Response and physiological connectivity of the mesolimbic system , 2005, NeuroImage.

[49]  Piotr Synak,et al.  Extracting Emotions from Music Data , 2005, ISMIS.

[50]  Tao Li,et al.  Music genre classification with taxonomy , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[51]  I. Peretz,et al.  Impaired recognition of scary music following unilateral temporal lobe excision. , 2005, Brain : a journal of neurology.

[52]  Katharina Morik,et al.  Automatic Feature Extraction for Classifying Audio Data , 2005, Machine Learning.

[53]  P. Laukka,et al.  Expression, Perception, and Induction of Musical Emotions: A Review and a Questionnaire Study of Everyday Listening , 2004 .

[54]  Beth Logan,et al.  Semantic analysis of song lyrics , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[55]  Isabelle Peretz,et al.  Singing in the Brain: Insights from Cognitive Neuropsychology , 2004 .

[56]  Lie Lu,et al.  Automatic mood detection from acoustic music data , 2003, ISMIR.

[57]  Tao Li,et al.  Detecting emotion in music , 2003, ISMIR.

[58]  Tao Li,et al.  Factors in automatic musical genre classification of audio signals , 2003, 2003 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (IEEE Cat. No.03TH8684).

[59]  C. Krumhansl,et al.  Measuring and Modeling Real-Time Responses to Music: The Dynamics of Tonality Induction , 2003, Perception.

[60]  Laurel J. Trainor,et al.  Preference for Sensory Consonance in 2- and 4-Month-Old Infants , 2002 .

[61]  J. Panksepp,et al.  Emotional sounds and the brain: the neuro-affective foundations of musical appreciation , 2002, Behavioural Processes.

[62]  R. Zatorre,et al.  Intensely pleasurable responses to music correlate with activity in brain regions implicated in reward and emotion , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[63]  I. Peretz,et al.  A developmental study of the affective value of tempo and mode in music , 2001, Cognition.

[64]  Roddy Cowie,et al.  FEELTRACE: an instrument for recording perceived emotion in real time , 2000 .

[65]  Julius O. Smith,et al.  Bark and ERB bilinear transforms , 1999, IEEE Trans. Speech Audio Process..

[66]  I. Peretz,et al.  Singing in the Brain: Independence of Lyrics and Tunes , 1998 .

[67]  I. Peretz,et al.  Music and emotion: perceptual determinants, immediacy, and isolation after brain damage , 1998, Cognition.

[68]  C. Krumhansl An exploratory study of musical emotions and psychophysiology. , 1997, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[69]  R. Thayer The Origin of Everyday Moods: Managing Energy, Tension, and Stress , 1996 .

[70]  John Saunders,et al.  Real-time discrimination of broadcast speech/music , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[71]  C. Krumhansl A Perceptual Analysis of Mozart's Piano Sonata K. 282: Segmentation, Tension, and Musical Ideas , 1996 .

[72]  Fred Lerdahl,et al.  Calculating Tonal Tension , 1996 .

[73]  R. Daroff,et al.  An Anthropologist on Mars , 1995, Neurology.

[74]  Sean A. Spence,et al.  Descartes' Error: Emotion, Reason and the Human Brain , 1995 .

[75]  D. Cicchetti Emotion and Adaptation , 1993 .

[76]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[77]  B. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[78]  P. Ekman An argument for basic emotions , 1992 .

[79]  T. Landauer,et al.  Latent semantic analysis , 2008, Scholarpedia.

[80]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[81]  Peter Kivy,et al.  Sound Sentiment: An Essay on the Musical Emotions, including the complete text of The Corded Shell , 1989 .

[82]  Seiji Inokuchi,et al.  Sentiment extraction in music , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[83]  James D. Johnston,et al.  Transform coding of audio signals using perceptual noise criteria , 1988, IEEE J. Sel. Areas Commun..

[84]  B. Kedem,et al.  Spectral analysis and discrimination by zero-crossings , 1986, Proceedings of the IEEE.

[85]  J. Russell,et al.  Concept of Emotion Viewed From a Prototype Perspective , 1984 .

[86]  P. Kleinginna,et al.  A categorized list of motivation definitions, with a suggestion for a consensual definition , 1981 .

[87]  J. Russell A circumplex model of affect. , 1980 .

[88]  E. Zwicker,et al.  Analytical expressions for critical‐band rate and critical bandwidth as a function of frequency , 1980 .

[89]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[90]  Svante Wold,et al.  Pattern recognition by means of disjoint principal components models , 1976, Pattern Recognit..

[91]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[92]  L. Wedin,et al.  A multidimensional study of perceptual-emotional qualities in music. , 1972, Scandinavian journal of psychology.

[93]  Alfred Pike A Phenomenological Analysis of Emotional Experience in Music , 1972 .

[94]  P. Farnsworth A STUDY OF THE HEVNER ADJECTIVE LIST , 1954 .

[95]  J. Cornfield,et al.  A method of estimating comparative rates from clinical data; applications to cancer of the lung, breast, and cervix. , 1951, Journal of the National Cancer Institute.

[96]  K. Hevner Experimental studies of the elements of expression in music , 1936 .

[97]  J. Mercer Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[98]  J. Stainer,et al.  The Emotions , 1922, Nature.

[99]  Perfecto Herrera,et al.  Automatic Detection of Emotion in Music: Interaction with Emotionally Sensitive Machines , 2018 .

[100]  Enric Guaus i Termens Audio content processing for automatic music genre classification: descriptors, databases, and classifiers , 2010 .

[101]  Cory McKay,et al.  Automatic music classification with jmir , 2010 .

[102]  Emilia Gómez,et al.  Music classification using high-level models , 2010 .

[103]  Menno van Zaanen,et al.  Automatic Mood Classification Using TF*IDF Based on Lyrics , 2010, ISMIR.

[104]  Björn W. Schuller,et al.  Determination of Nonprototypical Valence and Arousal in Popular Music: Features and Performances , 2010, EURASIP J. Audio Speech Music. Process..

[105]  R. Fisher On the Interpretation of χ2 from Contingency Tables, and the Calculation of P , 2010 .

[106]  Jeffrey J. Scott,et al.  State of the Art Report: Music Emotion Recognition: A State of the Art Review , 2010, ISMIR.

[107]  Petri Toiviainen,et al.  Prediction of Multidimensional Emotional Ratings in Music from Audio Using Multivariate Regression Models , 2009, ISMIR.

[108]  C. Laurier Estimating Tonal Tension from Audio Content , 2009 .

[109]  Joan Serrà,et al.  MUSIC TYPE GROUPERS (MTG): GENERIC MUSIC CLASSIFICATION ALGORITHMS , 2009 .

[110]  Christopher Raphael,et al.  Relationships Between Lyrics and Melody in Popular Music , 2009, ISMIR.

[111]  Wolfgang Nejdl,et al.  Music Mood and Theme Classification - a Hybrid Approach , 2009, ISMIR.

[112]  Perfecto Herrera,et al.  Mood Cloud 2.0: Music Mood Browsing based on Social Networks , 2009 .

[113]  Alessandro Bozzon,et al.  Pharos: An Audiovisual Search Platform using Music Information Retrieval Techniques , 2009 .

[114]  Seungmin Rho,et al.  SMERS: Music Emotion Recognition Using Support Vector Regression , 2009, ISMIR.

[115]  Joan Serrà,et al.  Music Mood Representations from Social Tags , 2009, ISMIR.

[116]  Yajie Hu,et al.  Lyric-based Song Emotion Detection with Affective Lexicon and Fuzzy Clustering Method , 2009, ISMIR.

[117]  Mohamed Sordo,et al.  ISMIR 2008 – Session 2c – Knowledge Representation, Tags, Metadata THE QUEST FOR MUSICAL GENRES: DO THE EXPERTS AND THE WISDOM OF CROWDS AGREE? , 2022 .

[118]  George Tzanetakis,et al.  MARSYAS-0.2: A Case Study in Implementing Music Information Retrieval Systems , 2008 .

[119]  Perfecto Herrera,et al.  Mood Cloud : A Real-Time Music Mood Visualization Tool , 2008 .

[120]  Andreas Rauber,et al.  Rhyme and Style Features for Musical Genre Classification by Song Lyrics , 2008, ISMIR.

[121]  Youngmoo E. Kim,et al.  MoodSwings: A Collaborative Game for Music Mood Label Collection , 2008, ISMIR.

[122]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[123]  Janto Skowronek,et al.  A Demonstrator for Automatic Music Mood Estimation , 2007, ISMIR.

[124]  Òscar Celma,et al.  Annotating Music Collections: How Content-Based Similarity Helps to Propagate Labels , 2007, ISMIR.

[125]  Perfecto Herrera,et al.  A computational approach to classify music by emotion , 2007 .

[126]  Petri Toiviainen,et al.  MIR in Matlab (II): A Toolbox for Musical Feature Extraction from Audio , 2007, ISMIR.

[127]  Mark B. Sandler,et al.  A Semantic Space for Music Derived from Social Tags , 2007, ISMIR.

[128]  O. Meyers A mood-based music classification and exploration system , 2007 .

[129]  Charles Nussbaum,et al.  The Musical Representation: Meaning, Ontology, and Emotion , 2007 .

[130]  Michael I. Mandel,et al.  AUDIO MUSIC MOOD CLASSIFICATION USING SUPPORT VECTOR MACHINE , 2007 .

[131]  Bart Baesens,et al.  Using Rule Extraction to Improve the Comprehensibility of Predictive Models , 2006 .

[132]  Morwaread Farbood,et al.  A quantitative, parametric model of musical tension , 2006 .

[133]  Nicola Orio,et al.  Music Retrieval: A Tutorial and Review , 2006, Found. Trends Inf. Retr..

[134]  Daniel J. Levitin,et al.  This is your brain on music : the science of a human obsession , 2006 .

[135]  Akinori Ito,et al.  Lyrics Recognition from a Singing Voice Based on Finite State Automaton for Music Information Retrieval , 2005, ISMIR.

[136]  G. Widmer,et al.  EVALUATION OF FREQUENTLY USED AUDIO FEATURES FOR CLASSIFICATION OF MUSIC INTO PERCEPTUAL CATEGORIES , 2005 .

[137]  Xavier Serra,et al.  SIMAC: semantic interaction with music audio contents , 2005 .

[138]  Katharina Morik,et al.  A Benchmark Dataset for Audio Classification and Clustering , 2005, ISMIR.

[139]  François Pachet,et al.  Improving Timbre Similarity : How high’s the sky ? , 2004 .

[140]  Dan Yang,et al.  Disambiguating Music Emotion Using Software Agents , 2004, ISMIR.

[141]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[142]  J. Sloboda,et al.  Psychological perspectives on music and emotion , 2001 .

[143]  A. Gabrielsson Emotions in strong experiences with music. , 2001 .

[144]  K. Scherer,et al.  EMOTIONAL EFFECTS OF MUSIC: PRODUCTION RULES , 2001 .

[145]  S. Davies Philosophical perspectives on music's expressiveness , 2001 .

[146]  Özgür Izmirli,et al.  Using a Spectral Flatness Based Feature for Audio Segmentation and Retrieval , 2000, ISMIR.

[147]  Beth Logan,et al.  Mel Frequency Cepstral Coefficients for Music Modeling , 2000, ISMIR.

[148]  Anna Wierzbicka,et al.  Emotions Across Languages and Cultures: Diversity and Universals: Defining emotion concepts: discovering “cognitive scenarios” , 1999 .

[149]  Narcotics Anonymous,et al.  Printed in the United States of America , 1998 .

[150]  W. Sethares Tuning, Timbre, Spectrum, Scale , 1998 .

[151]  Klaus R. Scherer,et al.  Emotion expression in speech and music , 1991 .

[152]  R. Thayer The biopsychology of mood and arousal , 1989 .

[153]  S. Tomkins AFFECT AS AMPLIFICATION: SOME MODIFICATIONS IN THEORY , 1980 .

[154]  Gerard Salton,et al.  The SMART Retrieval System—Experiments in Automatic Document Processing , 1971 .

[155]  Daniel P. W. Ellis,et al.  Please Scroll down for Article Journal of New Music Research a Web-based Game for Collecting Music Metadata a Web-based Game for Collecting Music Metadata , 2022 .