Mining the Correlation between Lyrical and Audio Features and the Emergence of Mood

Understanding the mood of music holds great potential for recommendation and genre identification problems. Unfortunately, hand-annotating music with mood tags is usually an expensive, time-consuming and subjective process, to such an extent that automatic mood recognition methods are required. In this paper we present a new unsupervised learning approach for mood recognition, based on the lyrics and the audio of a song. Our system thus eliminates the need for ground truth mood annotations, even for training the system. We hypothesize that lyrics and audio are both partially determined by the mood, and that there are no other strong common effects affecting these aspects of music. Based on this assumption, mood can be detected by performing a multi-modal analysis, identifying what lyrics and audio have in common. We demonstrate the effectiveness of this using Canonical Correlation Analysis, and confirm our hypothesis in a subsequent analysis of the results.

[1]  Mert Bay,et al.  The 2007 MIREX Audio Mood Classification Task: Lessons Learned , 2008, ISMIR.

[2]  J. Stephen Downie,et al.  When Lyrics Outperform Audio for Music Mood Classification: A Feature Analysis , 2010, ISMIR.

[3]  Peter Knees,et al.  USING BLOCK-LEVEL FEATURES FOR GENRE CLASSIFICATION , TAG CLASSIFICATION AND MUSIC SIMILARITY ESTIMATION , 2010 .

[4]  Yi-Hsuan Yang,et al.  Toward Multi-modal Music Emotion Classification , 2008, PCM.

[5]  H. Hotelling.  Relations Between Two Sets of Variates , 1936 .

[6]  Yajie Hu,et al.  Lyric-based Song Emotion Detection with Affective Lexicon and Fuzzy Clustering Method , 2009, ISMIR.

[7]  Tijl De Bie,et al.  Eigenproblems in Pattern Recognition , 2005 .

[8]  J. Stephen Downie,et al.  Improving mood classification in music digital libraries by combining lyrics and audio , 2010, JCDL '10.

[9]  M. Bradley,et al.  Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings , 1999 .

[10]  Gert R. G. Lanckriet,et al.  Identifying Words that are Musically Meaningful , 2007, ISMIR.

[11]  Jens Grivolla,et al.  Multimodal Music Mood Classification Using Audio and Lyrics , 2008, 2008 Seventh International Conference on Machine Learning and Applications.

[12]  J. Russell.  A circumplex model of affect. , 1980 .

[13]  Johan Bollen,et al.  Between Conjecture and Memento: Shaping A Collective Emotional Perception of the Future , 2008, AAAI Spring Symposium: Emotion, Personality, and Social Behavior.

[14]  Geoffroy Peeters,et al.  MIREX-2010 SINGLE-LABEL AND MULTI-LABEL CLASSIFICATION TASKS : IRCAMCLASSIFICATION 09 SUBMISSION , 2010 .

[15]  Hui He,et al.  Language Feature Mining for Music Emotion Classification via Supervised Learning from Lyrics , 2008, ISICA.