Lexicon information in neural sentiment analysis: a multi-task learning approach

This paper explores the use of multi-task learning (MTL) for incorporating external knowledge in neural models. Specifically, we show how MTL can enable a BiLSTM sentiment classifier to incorporate information from sentiment lexicons. Our MTL set-up is shown to improve model performance (compared to a single-task set-up) on both English and Norwegian sentence-level sentiment datasets. The paper also introduces a new sentiment lexicon for Norwegian.

[1]  Barbara Plank,et al.  When is multitask learning effective? Semantic sequence prediction under varying data conditions , 2016, EACL.

[2]  Finn Årup Nielsen,et al.  A New ANEW: Evaluation of a Word List for Sentiment Analysis in Microblogs , 2011, #MSM.

[3]  Erik Velldal,et al.  NoReC: The Norwegian Review Corpus , 2017, LREC.

[4]  Roser Morante,et al.  ConanDoyle-neg: Annotation of negation cues and their scope in Conan Doyle stories , 2012, LREC.

[5]  Isabelle Augenstein,et al.  Multi-Task Learning of Keyphrase Boundary Classification , 2017, ACL.

[6]  Michael L. Littman,et al.  Measuring praise and criticism: Inference of semantic orientation from association , 2003, TOIS.

[7]  Anis Yazidi,et al.  Building sentiment Lexicons applying graph theory on information from three Norwegian thesauruses , 2014, NIK.

[8]  Maite Taboada,et al.  Methods for Creating Semantic Orientation Dictionaries , 2006, LREC.

[9]  Murhaf Fares,et al.  Word vectors, reuse, and replicability: Towards a community repository of large-text resources , 2017, NODALIDA.

[10]  Rich Caruana,et al.  Multitask Learning: A Knowledge-Based Source of Inductive Bias , 1993, ICML.

[11]  Patrik Lambert,et al.  Attention and Lexicon Regularized LSTM for Aspect-based Sentiment Analysis , 2019, ACL.

[12]  Nanyun Peng,et al.  Multi-task Domain Adaptation for Sequence Tagging , 2016, Rep4NLP@ACL.

[13]  Kang Liu,et al.  Book Review: Sentiment Analysis: Mining Opinions, Sentiments, and Emotions by Bing Liu , 2015, CL.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[16]  Dirk Hovy,et al.  Learning part-of-speech taggers with inter-annotator agreement loss , 2014, EACL.

[17]  Hinrich Schütze,et al.  Sentiment Relevance , 2013, ACL.

[18]  Iryna Gurevych,et al.  Sentence and Expression Level Annotation of Opinions in User-Generated Discourse , 2010, ACL.

[19]  Andrea Esuli,et al.  SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining , 2006, LREC.

[20]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[21]  Johannes Bjerva,et al.  Will my auxiliary tagging task help? Estimating Auxiliary Tasks Effectivity in Multi-Task Learning , 2017, NODALIDA.

[22]  Emiliano Raúl Guevara,et al.  NoWaC: a large web-based corpus for Norwegian , 2010, WAC@NAACL-HLT.

[23]  Barbara Plank.  Keystroke dynamics as signal for shallow syntactic parsing , 2016, COLING.

[24]  Claire Cardie,et al.  Deep Recursive Neural Networks for Compositionality in Language , 2014, NIPS.

[25]  Bonggun Shin,et al.  Lexicon Integrated CNN Models with Attention for Sentiment Analysis , 2016, WASSA@EMNLP.

[26]  Sabine Schulte im Walde,et al.  Improving Verb Metaphor Detection by Propagating Abstractness to Words, Phrases and Individual Senses , 2017 .

[27]  M. Bradley,et al.  Affective Norms for English Words (ANEW): Instruction Manual and Affective Ratings , 1999 .

[28]  Soo-Min Kim,et al.  Determining the Sentiment of Opinions , 2004, COLING.

[29]  Philip J. Stone,et al.  The general inquirer: A computer system for content analysis and retrieval based on the sentence as a unit of information , 2007 .

[30]  Min Yang,et al.  Sentiment Lexicon Enhanced Attention-Based LSTM for Sentiment Classification , 2018, AAAI.

[31]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[32]  Saif Mohammad,et al.  NRC-Canada: Building the State-of-the-Art in Sentiment Analysis of Tweets , 2013, *SEMEVAL.

[33]  Anders Søgaard,et al.  Deep multi-task learning with low level tasks supervised at lower layers , 2016, ACL.

[34]  Yue Zhang,et al.  Context-Sensitive Lexicon Features for Neural Sentiment Analysis , 2016, EMNLP.

[35]  Yi Liu,et al.  A Multi-sentiment-resource Enhanced Attention Network for Sentiment Classification , 2018, ACL.

[36]  Janyce Wiebe,et al.  Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis , 2005, HLT.

[37]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[38]  Alexandros Potamianos,et al.  Attention-based Conditioning Methods for External Knowledge Integration , 2019, ACL.

[39]  Joachim Bingel,et al.  Identifying beneficial task relations for multi-task learning in deep neural networks , 2017, EACL.

[40]  Anis Yazidi,et al.  Constructing Sentiment Lexicons in Norwegian from a Large Text Corpus , 2014, 2014 IEEE 17th International Conference on Computational Science and Engineering.

[41]  Isabelle Augenstein,et al.  Multi-Task Learning of Pairwise Sequence Classification Tasks over Disparate Label Spaces , 2018, NAACL.

[42]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[43]  Maite Taboada,et al.  A review corpus annotated for negation, speculation and their scope , 2012, LREC.

[44]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[45]  Murhaf Fares,et al.  Transfer and Multi-Task Learning for Noun-Noun Compound Interpretation , 2018, EMNLP.

[46]  Erik Velldal,et al.  Annotating evaluative sentences for sentiment analysis: a dataset for Norwegian , 2019, NODALIDA.

[47]  Roman Klinger,et al.  IMS at EmoInt-2017: Emotion Intensity Prediction with Affective Norms, Automatically Extended Resources and Deep Learning , 2017, WASSA@EMNLP.

[48]  Saif Mohammad,et al.  CROWDSOURCING A WORD–EMOTION ASSOCIATION LEXICON , 2013, Comput. Intell..

[49]  Xuanjing Huang,et al.  A Lexicon-Based Supervised Attention Model for Neural Sentiment Analysis , 2018, COLING.