Semi-Supervised Method for Multi-category Emotion Recognition in Tweets

Each tweet is limited to 140 characters. This constraint surprisingly makes Twitter a more spontaneous platform to express our emotions. Detecting emotions and correctly classifying them automatically is an increasingly important task if we want to understand how large groups of people feel about an event or relevant topic. However, constructing supervised classifiers can be a daunting task because of the high manual annotation costs. We propose constructing emotion classifiers with a minimal amount of initial knowledge (e.g. A general-purpose emotion lexicon) and using a semi-supervised learning method to extend it to correctly detect more emotional tweets within a specific domain. Additionally, we show that our algorithm, Balanced Weighted Voting (or BWV) is able to overcome the imbalanced distribution of emotions in the initial labeled data. Our validation experiments show that BWV improves the performance of three initial classifiers, at least in the specific domain of sports. Furthermore, its comparison with other two learning strategies reveals its superiority in terms of macro F1-score, as well as more stable performance among different emotion categories.

[1]  P. Ekman An argument for basic emotions , 1992 .

[2]  R Swan,et al.  BUILDING THE STATE OF THE ART , 1997 .

[3]  Eibe Frank,et al.  Naive Bayes for Text Classification with Unbalanced Classes , 2006, PKDD.

[4]  Carlo Strapparava,et al.  WordNet Affect: an Affective Extension of WordNet , 2004, LREC.

[5]  Jonathan Klein,et al.  Computers that recognise and respond to user emotion: theoretical and practical implications , 2002, Interact. Comput..

[6]  Vladan Devedzic,et al.  Synesketch: An Open Source Library for Sentence-Based Emotion Recognition , 2013, IEEE Transactions on Affective Computing.

[7]  Sanda M. Harabagiu,et al.  EmpaTweet: Annotating and Detecting Emotions on Twitter , 2012, LREC.

[8]  Claudiu Cristian Musat,et al.  Fine-Grained Emotion Recognition in Olympic Tweets Based on Human Computation , 2013, WASSA@NAACL-HLT.

[9]  Nathalie Japkowicz,et al.  The Class Imbalance Problem: Significance and Strategies , 2000 .

[10]  Mitsuru Ishizuka,et al.  Affect Analysis Model: novel rule-based approach to affect sensing from text , 2010, Natural Language Engineering.

[11]  Sunghwan Mac Kim,et al.  Evaluation of Unsupervised Emotion Models to Textual Affect Recognition , 2010, HLT-NAACL 2010.

[12]  Saif Mohammad,et al.  #Emotional Tweets , 2012, *SEMEVAL.

[13]  Likun Qiu,et al.  SELC: a self-supervised model for sentiment classification , 2009, CIKM.

[14]  Saif Mohammad,et al.  CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON , 2013, Comput. Intell..

[15]  Gilad Mishne,et al.  MoodViews: Tools for Blog Mood Analysis , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[16]  Saif Mohammad,et al.  NRC-Canada: Building the State-of-the-Art in Sentiment Analysis of Tweets , 2013, *SEMEVAL.

[17]  Ellen Riloff,et al.  Bootstrapped Learning of Emotion Hashtags #hashtags4you , 2013, WASSA@NAACL-HLT.

[18]  Bing Liu,et al.  Sentiment Analysis and Opinion Mining , 2012, Synthesis Lectures on Human Language Technologies.

[19]  Erik Cambria,et al.  The Hourglass of Emotions , 2011, COST 2102 Training School.

[20]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[21]  Dipankar Das,et al.  Enriching SenticNet Polarity Scores through Semi-Supervised Fuzzy Clustering , 2012, 2012 IEEE 12th International Conference on Data Mining Workshops.

[22]  George Bojadziev,et al.  Fuzzy Sets, Fuzzy Logic, Applications , 1996, Advances in Fuzzy Systems - Applications and Theory.

[23]  Amit P. Sheth,et al.  Harnessing Twitter "Big Data" for Automatic Emotion Identification , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[24]  Vlado Keselj,et al.  Using Google n-Grams to Expand Word-Emotion Association Lexicon , 2013, CICLing.

[25]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[26]  Daniele Quercia,et al.  The Social World of Twitter: Topics, Geography, and Emotions , 2012, ICWSM.

[27]  Stan Szpakowicz,et al.  Hierarchical versus Flat Classification of Emotions in Text , 2010, HLT-NAACL 2010.

[28]  Carlo Strapparava,et al.  Learning to identify emotions in text , 2008, SAC '08.

[29]  Eric Gilbert,et al.  VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text , 2014, ICWSM.

[30]  P. Wilson,et al.  The Nature of Emotions , 2012 .

[31]  Dipankar Das,et al.  Enhanced SenticNet with Affective Labels for Concept-Based Opinion Mining , 2013, IEEE Intelligent Systems.

[32]  K. Scherer What are emotions? And how can they be measured? , 2005 .

[33]  Munmun De Choudhury,et al.  Happy, Nervous or Surprised? Classification of Human Affective States in Social Media , 2012, ICWSM.

[34]  Mike Thelwall,et al.  Sentiment strength detection for the social web , 2012, J. Assoc. Inf. Sci. Technol..

[35]  Xiaojin Zhu,et al.  --1 CONTENTS , 2006 .

[36]  Pearl Pu,et al.  Prediction of Helpful Reviews Using Emotions Extraction , 2014, AAAI.

[37]  Michael L. Littman,et al.  Measuring praise and criticism: Inference of semantic orientation from association , 2003, TOIS.

[38]  Dietrich Klakow,et al.  Predictive Features in Semi-Supervised Learning for Polarity Classification and the Role of Adjectives , 2009, NODALIDA.

[39]  Stan Szpakowicz,et al.  Identifying Expressions of Emotion in Text , 2007, TSD.

[40]  Nancy Ide,et al.  Distant Supervision for Emotion Classification with Discrete Binary Values , 2013, CICLing.

[41]  Erik Cambria,et al.  EmoSenticSpace: A novel framework for affective common-sense reasoning , 2014, Knowl. Based Syst..

[42]  Lillian Lee,et al.  Opinion Mining and Sentiment Analysis , 2008, Found. Trends Inf. Retr..

[43]  Andrew Ortony,et al.  The Cognitive Structure of Emotions , 1988 .

[44]  Vaibhavi N Patodkar,et al.  Twitter as a Corpus for Sentiment Analysis and Opinion Mining , 2016 .

[45]  K. Scherer,et al.  The GRID meets the wheel: assessing emotional feeling via self-report , 2013 .