Fine-Grained Emotion Detection in Suicide Notes: A Thresholding Approach to Multi-Label Classification

We present a system to automatically identify emotion-carrying sentences in suicide notes and to detect the specific fine-grained emotion conveyed. With this system, we competed in Track 2 of the 2011 Medical NLP Challenge, 14 where the task was to distinguish between fifteen emotion labels, from guilt, sorrow, and hopelessness to hopefulness and happiness. Since a sentence can be annotated with multiple emotions, we designed a thresholding approach that enables assigning multiple labels to a single instance. We rely on the probability estimates returned by an SVM classifier and experimentally set thresholds on these probabilities. Emotion labels are assigned only if their probability exceeds a certain threshold and if the probability of the sentence being emotion-free is low enough. We show the advantages of this thresholding approach by comparing it to a naïve system that assigns only the most probable label to each test sentence, and to a system trained on emotion-carrying sentences only.

[1]  Carlo Strapparava,et al.  WordNet Affect: an Affective Extension of WordNet , 2004, LREC.

[2]  Andrea Esuli,et al.  SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining , 2010, LREC.

[3]  Patricio Martínez-Barco,et al.  Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (WASSA 2.011) , 2011, WASSA@ACL.

[4]  Saif Mohammad,et al.  Tracking Sentiment in Mail: How Genders Differ on Emotional Axes , 2011, WASSA@ACL.

[5]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[6]  John Pestian,et al.  Using Natural Language Processing to Classify Suicide Notes , 2008, BioNLP.

[7]  K. Bretonnel Cohen,et al.  Sentiment Analysis of Suicide Notes: A Shared Task , 2012, Biomedical informatics insights.

[8]  Diana Inkpen,et al.  Using sentiment orientation features for mood classification in blogs , 2009, 2009 International Conference on Natural Language Processing and Knowledge Engineering.

[9]  Ellen Riloff,et al.  Toward Plot Units: Automatic Affect State Analysis , 2010, HLT-NAACL 2010.

[10]  Wlodzislaw Duch,et al.  Clustering Semantic Spaces of Suicide Notes and Newsgroups Articles , 2009, BioNLP@HLT-NAACL.

[11]  Sunghwan Mac Kim,et al.  Evaluation of Unsupervised Emotion Models to Textual Affect Recognition , 2010, HLT-NAACL 2010.

[12]  Menno van Zaanen,et al.  Automatic Mood Classification Using TF*IDF Based on Lyrics , 2010, ISMIR.

[13]  Stan Szpakowicz,et al.  Hierarchical versus Flat Classification of Emotions in Text , 2010, HLT-NAACL 2010.

[14]  J. Russell A circumplex model of affect. , 1980 .

[15]  Andrés Montoyo,et al.  Detecting Implicit Expressions of Sentiment in Text Based on Commonsense Knowledge , 2011, WASSA@ACL.

[16]  P. Ekman Universals and cultural differences in facial expressions of emotion. , 1972 .