Word-level emotion distribution with two schemas for short text emotion classification

Abstract Understanding word-level emotion in terms of both category and intensity has always been considered an essential step in addressing text emotion classification tasks. Existing studies have mainly adopted the categorical lexicons that are tagged by predefined emotion taxonomies to link affective words with discrete emotions. However, in these lexicons, emotion tags are restricted to a specific set of basic emotions. Moreover, the emotional intensity is ignored, making these methods less flexible and less informative. This paper proposes a novel method to generate a word-level emotion distribution (WED) vector by incorporating domain knowledge and dimensional lexicon. The proposed method can link a word with more generic and fine-grained emotion taxonomies with quantitatively computed intensities. We propose two schemas to utilize the WED vector implicitly and explicitly to facilitate classification. The implicit approach implements a rule-based conversion strategy to augment the information in the label space. The explicit approach exploits WED as an emotional word embedding to enhance the sentiment feature. We conduct extensive experiments on seven multiclass datasets. The results indicate that both proposed schemas produce competitive results compared with the state-of-the-art baselines.

[1]  Jianxin Wu,et al.  Deep Label Distribution Learning With Label Ambiguity , 2016, IEEE Transactions on Image Processing.

[2]  Aijun An,et al.  A Comprehensive Analysis of Preprocessing for Word Representation Learning in Affective Tasks , 2020, ACL.

[3]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[4]  Ming Zhou,et al.  Sentiment Embeddings with Applications to Sentiment Analysis , 2016, IEEE Transactions on Knowledge and Data Engineering.

[5]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[6]  Aijun An,et al.  Learning Emotion-enriched Word Representations , 2018, COLING.

[7]  Cecilia Ovesdotter Alm,et al.  Emotional Sequencing and Development in Fairy Tales , 2005, ACII.

[8]  Wei Li,et al.  User reviews: Sentiment analysis using lexicon integrated two-channel CNN-LSTM​ family models , 2020, Appl. Soft Comput..

[9]  Navdeep Jaitly,et al.  Hybrid speech recognition with Deep Bidirectional LSTM , 2013, 2013 IEEE Workshop on Automatic Speech Recognition and Understanding.

[10]  Donghong Ji,et al.  Latent Emotion Memory for Multi-Label Emotion Classification , 2020, AAAI.

[11]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[12]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[13]  M. Bradley,et al.  Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings , 1999 .

[14]  J. Russell,et al.  The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology , 2005, Development and Psychopathology.

[15]  Tomas Mikolov,et al.  Advances in Pre-Training Distributed Word Representations , 2017, LREC.

[16]  Haoran Xie,et al.  Fast Supervised Topic Models for Short Text Emotion Detection , 2021, IEEE Transactions on Cybernetics.

[17]  Wei Xiang,et al.  Encoding Syntactic Dependency and Topical Information for Social Emotion Classification , 2019, SIGIR.

[18]  J. Russell,et al.  The Measurement of Affect, Mood, and Emotion: A Guide for Health-Behavioral Research , 2013 .

[19]  Xuan Zhang,et al.  Emotion Distribution Learning from Texts , 2016, EMNLP.

[20]  Meinel Christoph,et al.  ASEDS: Towards Automatic Social Emotion Detection System Using Facebook Reactions , 2018, 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS).

[21]  Raymond Y. K. Lau,et al.  Universal affective model for Readers' emotion classification over short texts , 2018, Expert Syst. Appl..

[22]  Ying Zhang,et al.  Text Emotion Distribution Learning via Multi-Task Convolutional Neural Network , 2018, IJCAI.

[23]  P. Ekman An argument for basic emotions , 1992 .

[24]  Yulan He,et al.  A Rule-Based Approach to Implicit Emotion Detection in Text , 2015, NLDB.

[25]  Aliaksandr Barushka,et al.  Fake consumer review detection using deep neural networks integrating word embeddings and emotion mining , 2020, Neural Computing and Applications.

[26]  Asif Ekbal,et al.  How Intense Are You? Predicting Intensities of Emotions and Sentiments using Stacked Ensemble [Application Notes] , 2020, IEEE Comput. Intell. Mag..

[27]  Gelareh Mohammadi,et al.  A Multi-Componential Approach to Emotion Recognition and the Effect of Personality , 2020, IEEE Transactions on Affective Computing.

[28]  Erik Cambria,et al.  The Hourglass Model Revisited , 2020, IEEE Intelligent Systems.

[29]  Erik Cambria,et al.  SenticNet 6: Ensemble Application of Symbolic and Subsymbolic AI for Sentiment Analysis , 2020, CIKM.

[30]  Polina Rozenshtein,et al.  ESTeR: Combining Word Co-occurrences and Word Associations for Unsupervised Emotion Detection , 2020, FINDINGS.

[31]  Xin Geng,et al.  Label Distribution Learning , 2013, 2013 IEEE 13th International Conference on Data Mining Workshops.

[32]  Yi-Ling Chen,et al.  DACNN: Dynamic Weighted Attention with Multi-channel Convolutional Neural Network for Emotion Recognition , 2020, 2020 21st IEEE International Conference on Mobile Data Management (MDM).

[33]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[34]  Jianfei Yu,et al.  Improving Multi-label Emotion Classification via Sentiment Classification with Dual Attention Transfer Network , 2018, EMNLP.

[35]  Daling Wang,et al.  Context-aware emotion cause analysis with multi-attention-based neural network , 2019, Knowl. Based Syst..

[36]  Iyad Rahwan,et al.  Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm , 2017, EMNLP.

[37]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[38]  Zhaoxia Wang,et al.  A review of emotion sensing: categorization models and algorithms , 2020, Multimedia Tools and Applications.

[39]  Erik Cambria,et al.  ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis , 2021, Future Gener. Comput. Syst..

[40]  U. Rajendra Acharya,et al.  Automated accurate speech emotion recognition system using twine shuffle pattern and iterative neighborhood component analysis techniques , 2021, Knowl. Based Syst..

[41]  Donglei Tang,et al.  Hidden topic-emotion transition model for multi-level social emotion detection , 2019, Knowl. Based Syst..

[42]  J. Russell,et al.  Evidence for a three-factor theory of emotions , 1977 .

[43]  Raymond Y. K. Lau,et al.  Stock Prediction via Sentimental Transfer Learning , 2018, IEEE Access.

[44]  K. Scherer,et al.  Evidence for universality and cultural variation of differential emotion response patterning. , 1994, Journal of personality and social psychology.

[45]  Chang Wang,et al.  An End-to-end Topic-Enhanced Self-Attention Network for Social Emotion Classification , 2020, WWW.

[46]  Zhiyuan Liu,et al.  A C-LSTM Neural Network for Text Classification , 2015, ArXiv.

[47]  Hod Lipson,et al.  Re-embedding words , 2013, ACL.

[48]  Erik Cambria,et al.  Benchmarking Multimodal Sentiment Analysis , 2017, CICLing.

[49]  Erik Cambria,et al.  Cognitive-inspired domain adaptation of sentiment lexicons , 2019, Inf. Process. Manag..

[50]  Carlo Strapparava,et al.  SemEval-2007 Task 14: Affective Text , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[51]  Erik Cambria,et al.  Financial Sentiment Analysis: An Investigation into Common Mistakes and Silver Bullets , 2020, COLING.

[52]  Yoshihiko Hayashi,et al.  Exploiting Narrative Context and A Priori Knowledge of Categories in Textual Emotion Classification , 2020, COLING.

[53]  E. Cambria,et al.  Predicting political sentiments of voters from Twitter in multi-party contexts , 2020, Appl. Soft Comput..

[54]  Erik Cambria,et al.  OntoSenticNet: A Commonsense Ontology for Sentiment Analysis , 2018, IEEE Intelligent Systems.

[55]  Luca Oneto,et al.  Technical analysis and sentiment embeddings for market trend prediction , 2019, Expert Syst. Appl..

[56]  Saif Mohammad,et al.  Obtaining Reliable Human Ratings of Valence, Arousal, and Dominance for 20,000 English Words , 2018, ACL.

[57]  Saif Mohammad,et al.  #Emotional Tweets , 2012, *SEMEVAL.

[58]  Robert E. Mercer,et al.  Multi-Channel Convolutional Neural Network for Twitter Emotion and Sentiment Recognition , 2019, NAACL.

[59]  Ming Zhou,et al.  Human-machine dialogue modelling with the fusion of word- and sentence-level emotions , 2020, Knowl. Based Syst..

[60]  Yue Zhang,et al.  Context-Sensitive Lexicon Features for Neural Sentiment Analysis , 2016, EMNLP.

[61]  Tong Zhang,et al.  Deep Pyramid Convolutional Neural Networks for Text Categorization , 2017, ACL.