Semi-supervised emotion lexicon expansion with label propagation and specialized word embeddings

There exist two main approaches to automatically extract affective orientation: lexicon-based and corpus-based. In this work, we argue that these two methods are compatible and show that combining them can improve the accuracy of emotion classifiers. In particular, we introduce a novel variant of the Label Propagation algorithm that is tailored to distributed word representations, we apply batch gradient descent to accelerate the optimization of label propagation and to make the optimization feasible for large graphs, and we propose a reproducible method for emotion lexicon expansion. We conclude that label propagation can expand an emotion lexicon in a meaningful way and that the expanded emotion lexicon can be leveraged to improve the accuracy of an emotion classifier.

[1]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[2]  Roland Schäfer,et al.  Building Large Corpora from the Web Using a New Efficient Tool Chain , 2012, LREC.

[3]  Carlo Strapparava,et al.  WordNet Affect: an Affective Extension of WordNet , 2004, LREC.

[4]  Carlo Strapparava,et al.  SemEval-2007 Task 14: Affective Text , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[5]  Elizabeth D. Liddy,et al.  EmoTweet-28: A Fine-Grained Emotion Corpus for Sentiment Analysis , 2016, LREC.

[6]  Peter D. Turney Domain and Function: A Dual-Space Model of Semantic Relations and Compositions , 2012, J. Artif. Intell. Res..

[7]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[8]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[9]  Thorsten Joachims,et al.  Transductive Inference for Text Classification using Support Vector Machines , 1999, ICML.

[10]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[11]  Mitsuru Ishizuka,et al.  Compositionality Principle in Recognition of Fine-Grained Emotions from Text , 2009, ICWSM.

[12]  Andrea Esuli,et al.  SentiWordNet: A High-Coverage Lexical Resource for Opinion Mining , 2006 .

[13]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[14]  E. Edgington Approximate Randomization Tests , 1969 .

[15]  Claire Cardie,et al.  39. Opinion mining and sentiment analysis , 2014 .

[16]  Edouard Grave,et al.  Colorless Green Recurrent Networks Dream Hierarchically , 2018, North American Chapter of the Association for Computational Linguistics.

[17]  Saif Mohammad,et al.  SemEval-2018 Task 1: Affect in Tweets , 2018, *SEMEVAL.

[18]  Z. Šidák Rectangular Confidence Regions for the Means of Multivariate Normal Distributions , 1967 .

[19]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[20]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[21]  Xiuzhen Zhang,et al.  Sentiment Analysis on Twitter through Topic-Based Lexicon Expansion , 2014, ADC.

[22]  Geoffrey Zweig,et al.  Linguistic Regularities in Continuous Space Word Representations , 2013, NAACL.

[23]  Saif Mohammad,et al.  CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON , 2013, Comput. Intell..

[24]  K. P. Chow,et al.  LCCT: A Semi-supervised Model for Sentiment Classification , 2015, NAACL.

[25]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[26]  Pablo Gervás,et al.  SentiSense: An easily scalable concept-based affective lexicon for sentiment analysis , 2012, LREC.

[27]  Carlo Strapparava,et al.  Learning to identify emotions in text , 2008, SAC '08.

[28]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[29]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[30]  Carlo Strapparava,et al.  The Affective Weight of Lexicon , 2006, LREC.

[31]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[32]  Saif Mohammad,et al.  IEST: WASSA-2018 Implicit Emotions Shared Task , 2018, WASSA@EMNLP.

[33]  Hod Lipson,et al.  Re-embedding words , 2013, ACL.

[34]  Ming Zhou,et al.  Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification , 2014, ACL.

[35]  M. Bradley,et al.  Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings , 1999 .

[36]  Cecilia Ovesdotter Alm,et al.  Emotions from Text: Machine Learning for Text-based Emotion Prediction , 2005, HLT.

[37]  Sepp Hochreiter,et al.  Untersuchungen zu dynamischen neuronalen Netzen , 1991 .

[38]  Emmanuel Dupoux,et al.  Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies , 2016, TACL.

[39]  Bruce Bowe The Face of Emotion , 1985 .

[40]  Peter D. Turney Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews , 2002, ACL.

[41]  Stan Szpakowicz,et al.  Identifying Expressions of Emotion in Text , 2007, TSD.

[42]  Quoc V. Le,et al.  Exploiting Similarities among Languages for Machine Translation , 2013, ArXiv.

[43]  J. Russell,et al.  An approach to environmental psychology , 1974 .

[44]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[45]  Zoubin Ghahramani,et al.  Learning from labeled and unlabeled data with label propagation , 2002 .

[46]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[47]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[48]  P. Ekman An argument for basic emotions , 1992 .

[49]  Roland Schäfer,et al.  Processing and querying large web corpora with the COW14 architecture , 2015 .

[50]  K. Scherer,et al.  Evidence for universality and cultural variation of differential emotion response patterning. , 1994, Journal of personality and social psychology.

[51]  Saif Mohammad,et al.  Using Hashtags to Capture Fine Emotion Categories from Tweets , 2015, Comput. Intell..

[52]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[53]  Miles Osborne,et al.  The Edinburgh Twitter Corpus , 2010, HLT-NAACL 2010.

[54]  Mohammad Soleymani,et al.  A Multimodal Database for Affect Recognition and Implicit Tagging , 2012, IEEE Transactions on Affective Computing.

[55]  Alistair Kennedy,et al.  SENTIMENT CLASSIFICATION of MOVIE REVIEWS USING CONTEXTUAL VALENCE SHIFTERS , 2006, Comput. Intell..

[56]  Diana Inkpen,et al.  Using a Heterogeneous Dataset for Emotion Analysis in Text , 2011, Canadian Conference on AI.

[57]  Sabine Bergler,et al.  When Specialists and Generalists Work Together: Overcoming Domain Dependence in Sentiment Tagging , 2008, ACL.

[58]  R. Plutchik A GENERAL PSYCHOEVOLUTIONARY THEORY OF EMOTION , 1980 .

[59]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[60]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[61]  Saif Mohammad,et al.  Determining Word-Emotion Associations from Tweets by Multi-label Classification , 2016, 2016 IEEE/WIC/ACM International Conference on Web Intelligence (WI).

[62]  J. Kruskal On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[63]  Aapo Hyvärinen,et al.  Noise-Contrastive Estimation of Unnormalized Statistical Models, with Applications to Natural Image Statistics , 2012, J. Mach. Learn. Res..