Neural transition model for aspect-based sentiment triplet extraction with triplet memory

Abstract The aspect-based sentiment triplet extraction (ASTE), as a complete sentiment analysis task, aims to recognize the aspect term, the opinion expression, and the sentiment polarity in a sentence. Current state-of-the-art ASTE models employ a joint extracting scheme for better task improvements. However, how to better solve the triplet overlap issues in the task, and effectively model the mutual interactions between the triplet structures remain challenging. In this work, we explore a neural transition model for end-to-end ASTE. We model the triplet prediction as a graph structure, based on which we implement a transition system with neural design. We further propose a triplet memory mechanism to fully leverage the underlying interactions from the previously recognized triplets relevant to the current parse. We experiment on the benchmark datasets, and the results show that our model achieved state-of-the-art performances against current baselines, meanwhile being more efficient on decoding. Further analysis is conducted to verify the effectiveness of our transition framework.

[1]  Zhen Huang,et al.  Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification , 2019, ACL.

[2]  Xin Li,et al.  A Unified Model for Opinion Target Extraction and Target Sentiment Prediction , 2018, AAAI.

[3]  Sinno Jialin Pan,et al.  Syntactically-Meaningful and Transferable Recursive Neural Networks for Aspect and Opinion Extraction , 2020, Computational Linguistics.

[4]  Xiaozhong Liu,et al.  Cold-Start Aware Deep Memory Network for Multi-Entity Aspect-Based Sentiment Analysis , 2019, IJCAI.

[5]  Jun Zhao,et al.  Extracting Relational Facts by an End-to-End Neural Model with Copy Mechanism , 2018, ACL.

[6]  Baowen Xu,et al.  Gated Hierarchical LSTMs for Target-Based Sentiment Analysis , 2018, Int. J. Softw. Eng. Knowl. Eng..

[7]  Donghong Ji,et al.  Boundaries and edges rethinking: An end-to-end neural model for overlapping entity relation extraction , 2020, Inf. Process. Manag..

[8]  Donghong Ji,et al.  Topic-Enhanced Capsule Network for Multi-Label Emotion Classification , 2020, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[9]  Bo Shen,et al.  ReMemNN: A novel memory neural network for powerful interaction in aspect-based sentiment analysis , 2020, Neurocomputing.

[10]  Stephen Clark,et al.  A Tale of Two Parsers: Investigating and Combining Graph-based and Transition-based Dependency Parsing , 2008, EMNLP.

[11]  Wenji Mao,et al.  Multi-Interactive Memory Network for Aspect Based Multimodal Sentiment Analysis , 2019, AAAI.

[12]  Luo Si,et al.  Knowing What, How and Why: A Near Complete Solution for Aspect-based Sentiment Analysis , 2019, AAAI.

[13]  Yan Zhou,et al.  A Span-based Joint Model for Opinion Target Extraction and Target Sentiment Classification , 2019, IJCAI.

[14]  Kiyoaki Shirai,et al.  PhraseRNN: Phrase Recursive Neural Network for Aspect-based Sentiment Analysis , 2015, EMNLP.

[15]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[16]  Yafeng Ren,et al.  Implicit Objective Network for Emotion Detection , 2019, NLPCC.

[17]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[18]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[19]  Ravindra Kumar,et al.  Aspect-based sentiment analysis using deep networks and stochastic optimization , 2019, Neural Computing and Applications.

[20]  Hongxia Jin,et al.  A Neural Transition-based Model for Nested Mention Recognition , 2018, EMNLP.

[21]  Donghong Ji,et al.  Improving Text Understanding via Deep Syntax-Semantics Communication , 2020, FINDINGS.

[22]  Chenliang Li,et al.  Dependency Graph Enhanced Dual-transformer Structure for Aspect-based Sentiment Classification , 2020, ACL.

[23]  Haris Papageorgiou,et al.  SemEval-2016 Task 5: Aspect Based Sentiment Analysis , 2016, *SEMEVAL.

[24]  Shuai Wang,et al.  Target-Sensitive Memory Networks for Aspect Sentiment Classification , 2018, ACL.

[25]  Donghong Ji,et al.  End-to-end Semantic Role Labeling with Neural Transition-based Model , 2021, AAAI.

[26]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[27]  Ting Liu,et al.  Aspect Level Sentiment Classification with Deep Memory Network , 2016, EMNLP.

[28]  Yangqiu Song,et al.  Neural Aspect and Opinion Term Extraction with Mined Rules as Weak Supervision , 2019, ACL.

[29]  Patrik Lambert,et al.  Attention and Lexicon Regularized LSTM for Aspect-based Sentiment Analysis , 2019, ACL.

[30]  Abeer Alsadoon,et al.  Deep Learning for Aspect-Based Sentiment Analysis: A Comparative Review , 2019, Expert Syst. Appl..

[31]  Philip Ogunbona,et al.  Modelling Context and Syntactical Features for Aspect-based Sentiment Analysis , 2020, ACL.

[32]  Kewei Tu,et al.  Bidirectional Transition-Based Dependency Parsing , 2019, AAAI.

[33]  Luyao Huang,et al.  Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence , 2019, NAACL.

[34]  Mehrdad Jalali,et al.  A Novel Large-scale Model for Real Time Sentiment Analysis Using Online Shop Reviews and Comments , 2020, ICCAI.

[35]  Donghong Ji,et al.  Latent Emotion Memory for Multi-Label Emotion Classification , 2020, AAAI.

[36]  Lu Xu,et al.  Position-Aware Tagging for Aspect Sentiment Triplet Extraction , 2020, EMNLP.

[37]  Yue Zhang,et al.  Joint Word Segmentation, POS-Tagging and Syntactic Chunking , 2016, AAAI.

[38]  Li Zhao,et al.  Attention-based LSTM for Aspect-level Sentiment Classification , 2016, EMNLP.

[39]  Siu Cheung Hui,et al.  Learning to Attend via Word-Aspect Associative Fusion for Aspect-based Sentiment Analysis , 2017, AAAI.

[40]  Yunfei Long,et al.  Dual memory network model for sentiment analysis of review text , 2020, Knowl. Based Syst..

[41]  Bo Tang,et al.  Automatic Aspect-Based Sentiment Analysis (AABSA) from Customer Reviews , 2020, AffCon@AAAI.

[42]  Donghong Ji,et al.  Mimic and Conquer: Heterogeneous Tree Structure Distillation for Syntactic NLP , 2020, FINDINGS.

[43]  Suresh Manandhar,et al.  SemEval-2014 Task 4: Aspect Based Sentiment Analysis , 2014, *SEMEVAL.

[44]  Ruifeng Xu,et al.  Context-aware Embedding for Targeted Aspect-based Sentiment Analysis , 2019, ACL.

[45]  Yue Zhang,et al.  Transition-Based Neural Word Segmentation , 2016, ACL.

[46]  Yue Zhang,et al.  Transition-Based Neural Word Segmentation Using Word-Level Features , 2018, J. Artif. Intell. Res..

[47]  Mirella Lapata,et al.  Summarizing Opinions: Aspect Extraction Meets Sentiment Prediction and They Are Both Weakly Supervised , 2018, EMNLP.

[48]  Yang Li,et al.  Learning multi-grained aspect target sequence for Chinese sentiment analysis , 2018, Knowl. Based Syst..

[49]  Jian-Huang Lai,et al.  Deep Mask Memory Network with Semantic Dependency and Context Moment for Aspect Level Sentiment Classification , 2019, IJCAI.

[50]  Donghong Ji,et al.  Extracting Entities and Events as a Single Task Using a Transition-Based Neural Model , 2019, IJCAI.

[51]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[52]  Ming Zhou,et al.  Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification , 2014, ACL.

[53]  Xiaokui Xiao,et al.  Coupled Multi-Layer Attentions for Co-Extraction of Aspect and Opinion Terms , 2017, AAAI.

[54]  Wei Lu,et al.  Learning Latent Opinions for Aspect-level Sentiment Classification , 2018, AAAI.

[55]  Jie Liu,et al.  Hierarchical Sequence Labeling Model for Aspect Sentiment Triplet Extraction , 2020, NLPCC.

[56]  Véronique Hoste,et al.  Towards an integrated pipeline for aspect-based sentiment analysis in various domains , 2017, WASSA@EMNLP.

[57]  Donghong Ji,et al.  Cross-Lingual Semantic Role Labeling With Model Transfer , 2020, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[58]  Noah A. Smith,et al.  Transition-Based Dependency Parsing with Stack Long Short-Term Memory , 2015, ACL.

[59]  Stephen Clark,et al.  A Fast Decoder for Joint Word Segmentation and POS-Tagging Using a Single Discriminative Model , 2010, EMNLP.

[60]  Shuai Wang,et al.  Lifelong Learning Memory Networks for Aspect Sentiment Classification , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[61]  Suresh Manandhar,et al.  SemEval-2015 Task 12: Aspect Based Sentiment Analysis , 2015, *SEMEVAL.

[62]  Donghong Ji,et al.  Enriching contextualized language model from knowledge graph for biomedical information extraction , 2020, Briefings Bioinform..

[63]  Gonçalo Simões,et al.  Morphosyntactic Tagging with a Meta-BiLSTM Model over Context Sensitive Token Encodings , 2018, ACL.

[64]  Donghong Ji,et al.  Cross-Lingual Semantic Role Labeling with High-Quality Translated Training Corpus , 2020, ACL.

[65]  Donghong Ji,et al.  Retrofitting Structure-aware Transformer Language Model for End Tasks , 2020, EMNLP.

[66]  Kepeng Xu,et al.  Target-Based Attention Model for Aspect-Level Sentiment Analysis , 2019, ICONIP.