SRL4ORL: Improving Opinion Role Labeling Using Multi-Task Learning with Semantic Role Labeling

For over 12 years, machine learning is used to extract opinion-holder-target structures from text to answer the question: Who expressed what kind of sentiment towards what?. However, recent neural approaches do not outperform the state-of-the-art feature-based model for Opinion Role Labelling (ORL). We suspect this is due to the scarcity of labelled training data and address this issue using different multi-task learning techniques with a related task which has substantially more data, i.e. Semantic Role Labelling (SRL). Despite difficulties of the benchmark MPQA corpus, we show that indeed the ORL model benefits from SRL knowledge.

[1]  Yann LeCun,et al.  Recurrent Orthogonal Networks and Long-Memory Tasks , 2016, ICML.

[2]  Pierre Nugues,et al.  A High-Performance Syntactic and Semantic Dependency Parser , 2010, COLING.

[3]  Josef Ruppenhofer,et al.  Effect Functors for Opinion Inference , 2016, LREC.

[4]  Zoubin Ghahramani,et al.  A Theoretically Grounded Application of Dropout in Recurrent Neural Networks , 2015, NIPS.

[5]  Michael Wiegand,et al.  Separating Actor-View from Speaker-View Opinion Expressions using Linguistic Features , 2016, HLT-NAACL.

[6]  Iryna Gurevych,et al.  Optimal Hyperparameters for Deep LSTM-Networks for Sequence Labeling Tasks , 2017, ArXiv.

[7]  F. Massey.  The Kolmogorov-Smirnov Test for Goodness of Fit , 1951 .

[8]  Anders Søgaard,et al.  Deep multi-task learning with low level tasks supervised at lower layers , 2016, ACL.

[9]  Claire Cardie,et al.  Investigating LSTMs for Joint Extraction of Opinion Entities and Relations , 2016, ACL.

[10]  Tom M. Mitchell,et al.  A Joint Sequential and Relational Model for Frame-Semantic Parsing , 2017, EMNLP.

[11]  Alan Ritter,et al.  Adversarial Learning for Neural Dialogue Generation , 2017, EMNLP.

[12]  Diego Marcheggiani,et al.  Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling , 2017, EMNLP.

[13]  Noah A. Smith,et al.  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2016, ACL 2016.

[14]  David Bamman,et al.  Adversarial Training for Relation Extraction , 2017, EMNLP.

[15]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.

[16]  Luke S. Zettlemoyer,et al.  Deep Semantic Role Labeling: What Works and What’s Next , 2017, ACL.

[17]  Janyce Wiebe,et al.  Benefactive/Malefactive Event and Writer Attitude Annotation , 2013, ACL.

[18]  Michael Wiegand,et al.  Opinion Holder and Target Extraction on Opinion Compounds - A Linguistic Approach , 2016, HLT-NAACL.

[19]  Claire Cardie,et al.  Opinion Mining with Deep Recurrent Neural Networks , 2014, EMNLP.

[20]  Claire Cardie,et al.  Joint Inference for Fine-grained Opinion Extraction , 2013, ACL.

[21]  Shafiq R. Joty,et al.  Fine-grained Opinion Mining with Recurrent Neural Networks and Word Embeddings , 2015, EMNLP.

[22]  Claire Cardie,et al.  Joint Modeling of Opinion Expression Extraction and Attribute Classification , 2014, Transactions of the Association for Computational Linguistics.

[23]  Joachim Bingel,et al.  Identifying beneficial task relations for multi-task learning in deep neural networks , 2017, EACL.

[24]  Yoshimasa Tsuruoka,et al.  A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks , 2016, EMNLP.

[25]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[26]  Meng Zhang,et al.  Adversarial Training for Unsupervised Bilingual Lexicon Induction , 2017, ACL.

[27]  Theresa Wilson.  Fine-grained subjectivity and sentiment analysis: recognizing the intensity, polarity, and attitudes of private states , 2008 .

[28]  W. Bruce Croft,et al.  Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2013 .

[29]  Claire Cardie,et al.  Joint Extraction of Entities and Relations for Opinion Recognition , 2006, EMNLP.

[30]  Janyce Wiebe,et al.  Sentiment Propagation via Implicature Constraints , 2014, EACL.

[31]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[32]  Richard Johansson,et al.  Relational Features in Fine-Grained Opinion Analysis , 2013, CL.

[33]  Swapna Somasundaran,et al.  Finding the Sources and Targets of Subjective Expressions , 2008, LREC.

[34]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[35]  Yann LeCun,et al.  Orthogonal RNNs and Long-Memory Tasks , 2016, ArXiv.

[36]  Michael Wiegand,et al.  Opinion Holder and Target Extraction based on the Induction of Verbal Categories , 2015, CoNLL.

[37]  Xuanjing Huang,et al.  Adversarial Multi-task Learning for Text Classification , 2017, ACL.

[38]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[39]  Hai Zhao,et al.  Adversarial Connective-exploiting Networks for Implicit Discourse Relation Classification , 2017, ACL.

[40]  Xuanjing Huang,et al.  Part-of-Speech Tagging for Twitter with Adversarial Neural Networks , 2017, EMNLP.

[41]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[42]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[43]  Xavier Carreras,et al.  Introduction to the CoNLL-2005 Shared Task: Semantic Role Labeling , 2005, CoNLL.

[44]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[45]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[46]  Young-Bum Kim,et al.  Adversarial Adaptation of Synthetic or Stale Data , 2017, ACL.

[47]  Wojciech Zaremba,et al.  An Empirical Exploration of Recurrent Network Architectures , 2015, ICML.

[48]  Preslav Nakov,et al.  Cross-language Learning with Adversarial Neural Networks , 2017, CoNLL.

[49]  Eduard Hovy,et al.  Extracting Opinions, Opinion Holders, and Topics Expressed in Online News Media Text , 2006 .

[50]  Wei Xu,et al.  End-to-end learning of semantic role labeling using recurrent neural networks , 2015, ACL.

[51]  Barbara Plank,et al.  When is multitask learning effective? Semantic sequence prediction under varying data conditions , 2016, EACL.

[52]  Xuanjing Huang,et al.  Adversarial Multi-Criteria Learning for Chinese Word Segmentation , 2017, ACL.