Enabling Deep Learning of Emotion With First-Person Seed Expressions

The computational treatment of emotion in natural language text remains relatively limited, and Arabic is no exception. This is partly due to lack of labeled data. In this work, we describe and manually validate a method for the automatic acquisition of emotion labeled data and introduce a newly developed data set for Modern Standard and Dialectal Arabic emotion detection focused at Robert Plutchik’s 8 basic emotion types. Using a hybrid supervision method that exploits first person emotion seeds, we show how we can acquire promising results with a deep gated recurrent neural network. Our best model reaches 70% F-score, significantly (i.e., 11%, p < 0.05) outperforming a competitive baseline. Applying our method and data on an external dataset of 4 emotions released around the same time we finalized our work, we acquire 7% absolute gain in F-score over a linear SVM classifier trained on gold data, thus validating our approach.

[1]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[2]  P. Ekman Universals and cultural differences in facial expressions of emotion. , 1972 .

[3]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[4]  R. Plutchik On emotion: The chicken-and-egg problem revisited , 1985 .

[5]  P. Ekman An argument for basic emotions , 1992 .

[6]  R. Plutchik The psychology and biology of emotion , 1994 .

[7]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[8]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[9]  Bo Pang,et al.  A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts , 2004, ACL.

[10]  Hiroshi Kanayama,et al.  Deeper Sentiment Analysis Using Machine Translation Technology , 2004, COLING.

[11]  Janyce Wiebe,et al.  Learning Subjective Language , 2004, CL.

[12]  Rada Mihalcea,et al.  Learning Multilingual Subjective Language via Cross-Lingual Projections , 2007, ACL.

[13]  Stan Szpakowicz,et al.  Identifying Expressions of Emotion in Text , 2007, TSD.

[14]  Xiaojun Wan,et al.  Using Bilingual Knowledge and Ensemble Techniques for Unsupervised Chinese Sentiment Analysis , 2008, EMNLP.

[15]  B. Alexandra,et al.  Rethinking Sentiment Analysis in the News: from Theory to Practice and back , 2009 .

[16]  Daniel Jurafsky,et al.  Distant supervision for relation extraction without labeled data , 2009, ACL.

[17]  English Corpora,et al.  Cross-Linguistic Sentiment Analysis: From English to Spanish , 2009 .

[18]  Thin Nguyen,et al.  Mood Patterns and Affective Lexicon Access in Weblogs , 2010, ACL.

[19]  Bing Liu Sentiment Analysis and Opinion Mining Opinion Mining , 2011 .

[20]  Ohad Shamir,et al.  Better Mini-Batch Algorithms via Accelerated Gradient Methods , 2011, NIPS.

[21]  Muhammad Abdul-Mageed,et al.  Subjectivity and Sentiment Analysis of Modern Standard Arabic , 2011, ACL.

[22]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[23]  Bing Liu,et al.  Sentiment Analysis and Opinion Mining , 2012, Synthesis Lectures on Human Language Technologies.

[24]  Amit P. Sheth,et al.  Harnessing Twitter "Big Data" for Automatic Emotion Identification , 2012, 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing.

[25]  Saif Mohammad,et al.  #Emotional Tweets , 2012, *SEMEVAL.

[26]  Munmun De Choudhury,et al.  Not All Moods Are Created Equal! Exploring Human Emotional States in Social Media , 2012, ICWSM.

[27]  Saif Mohammad,et al.  CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON , 2013, Comput. Intell..

[28]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[29]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[30]  Verena Rieser,et al.  An Arabic Twitter Corpus for Subjectivity and Sentiment Analysis , 2014, LREC.

[31]  Muhammad Abdul-Mageed,et al.  SAMAR: Subjectivity and sentiment analysis for Arabic social media , 2014, Comput. Speech Lang..

[32]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[33]  Saif Mohammad,et al.  Sentiment after Translation: A Case-Study on Arabic Social Media Posts , 2015, NAACL.

[34]  Yoshua Bengio,et al.  Gated Feedback Recurrent Neural Networks , 2015, ICML.

[35]  Amir F. Atiya,et al.  ASTD: Arabic Sentiment Tweets Dataset , 2015, EMNLP.

[36]  Svitlana Volkova,et al.  Inferring Perceived Demographics from User Emotional Tone and User-Environment Emotional Contrast , 2016, ACL.

[37]  Saif Mohammad,et al.  How Translation Alters Sentiment , 2016, J. Artif. Intell. Res..

[38]  Saif Mohammad,et al.  Capturing Reliable Fine-Grained Sentiment Associations by Crowdsourcing and Best–Worst Scaling , 2016, NAACL.

[39]  Jasy Suet Yan Liew,et al.  Exploring Fine-Grained Emotion Detection in Tweets , 2016, NAACL.

[40]  Muhammad Abdul-Mageed,et al.  Does ‘well-being’ translate on Twitter? , 2016, EMNLP.

[41]  Preslav Nakov,et al.  SemEval-2016 Task 4: Sentiment Analysis in Twitter , 2016, *SEMEVAL.

[42]  Yi Yang,et al.  Overcoming Language Variation in Sentiment Analysis with Social Attention , 2015, TACL.

[43]  Saif Mohammad,et al.  Emotion Intensities in Tweets , 2017, *SEMEVAL.

[44]  Muhammad Abdul-Mageed,et al.  EmoNet: Fine-Grained Emotion Detection with Gated Recurrent Neural Networks , 2017, ACL.

[45]  L. F. Barrett How Emotions Are Made: The Secret Life of the Brain , 2017 .

[46]  B. Mesquita,et al.  Doing emotions: The role of culture in everyday emotions , 2017 .

[47]  Hassan Alhuzali,et al.  Think Before Your Click: Data and Models for Adult Content in Arabic Twitter , 2018 .

[48]  Muhammad Abdul-Mageed Learning Subjective Language : Feature Engineered vs . Deep Models , 2018 .

[49]  Saif Mohammad,et al.  SemEval-2018 Task 1: Affect in Tweets , 2018, *SEMEVAL.

[50]  Muhammad Abdul-Mageed,et al.  You Tweet What You Speak: A City-Level Dataset of Arabic Dialects , 2018, LREC.

[51]  Muhammad Abdul-Mageed,et al.  Modeling Arabic subjectivity and sentiment in lexical space , 2017, Inf. Process. Manag..

[52]  V. Sharmila,et al.  Using Hashtags to Capture Fine Emotion Categories from Tweets , 2019 .