Exploiting BERT for End-to-End Aspect-based Sentiment Analysis

In this paper, we investigate the modeling power of contextualized embeddings from pre-trained language models, e.g. BERT, on the E2E-ABSA task. Specifically, we build a series of simple yet insightful neural baselines to deal with E2E-ABSA. The experimental results show that even with a simple linear classification layer, our BERT-based architecture can outperform state-of-the-art works. Besides, we also standardize the comparative study by consistently utilizing a hold-out validation dataset for model selection, which is largely ignored by previous works. Therefore, our work can serve as a BERT-based benchmark for E2E-ABSA.

[1]  Zhen Huang,et al.  Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification , 2019, ACL.

[2]  Houfeng Wang,et al.  Interactive Attention Networks for Aspect-Level Sentiment Classification , 2017, IJCAI.

[3]  Richard Socher,et al.  Learned in Translation: Contextualized Word Vectors , 2017, NIPS.

[4]  Yue Zhang,et al.  Attention Modeling for Targeted Sentiment , 2017, EACL.

[5]  Kathleen M. Carley,et al.  Parameterized Convolutional Neural Networks for Aspect Level Sentiment Classification , 2019, EMNLP.

[6]  Bing Liu,et al.  DOER: Dual Cross-Shared RNN for Aspect Term-Polarity Co-Extraction , 2019, ACL.

[7]  Tingting Wang,et al.  Deep Learning for Aspect-Level Sentiment Classification: Survey, Vision, and Challenges , 2019, IEEE Access.

[8]  Xin Li,et al.  Transformation Networks for Target-Oriented Sentiment Classification , 2018, ACL.

[9]  Guillaume Lample,et al.  Neural Architectures for Named Entity Recognition , 2016, NAACL.

[10]  Asif Ekbal,et al.  IARM: Inter-Aspect Relation Modeling with Memory Networks in Aspect-Based Sentiment Analysis , 2018, EMNLP.

[11]  Ankur Bapna,et al.  The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation , 2018, ACL.

[12]  Hao Li,et al.  Learning Explicit and Implicit Structures for Targeted Sentiment Analysis , 2019, EMNLP/IJCNLP.

[13]  Philip S. Yu,et al.  BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis , 2019, NAACL.

[14]  Bowen Zhou,et al.  A Structured Self-attentive Sentence Embedding , 2017, ICLR.

[15]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .

[16]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[17]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[18]  Lidong Bing,et al.  Recurrent Attention Network on Memory for Aspect Sentiment Analysis , 2017, EMNLP.

[19]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[20]  Haris Papageorgiou,et al.  SemEval-2016 Task 5: Aspect Based Sentiment Analysis , 2016, *SEMEVAL.

[21]  Wei Xu,et al.  Bidirectional LSTM-CRF Models for Sequence Tagging , 2015, ArXiv.

[22]  Honglei Guo,et al.  Learning to Detect Opinion Snippet for Aspect-Based Sentiment Analysis , 2019, CoNLL.

[23]  Suresh Manandhar,et al.  SemEval-2014 Task 4: Aspect Based Sentiment Analysis , 2014, *SEMEVAL.

[24]  Houfeng Wang,et al.  Joint Learning for Targeted Sentiment Analysis , 2018, EMNLP.

[25]  Ting Liu,et al.  Aspect Level Sentiment Classification with Deep Memory Network , 2016, EMNLP.

[26]  Tao Shen,et al.  DiSAN: Directional Self-Attention Network for RNN/CNN-free Language Understanding , 2017, AAAI.

[27]  Qiang Yang,et al.  Exploiting Coarse-to-Fine Task Transfer for Aspect-level Sentiment Classification , 2018 .

[28]  Yang Liu,et al.  Fine-tune BERT for Extractive Summarization , 2019, ArXiv.

[29]  Benjamin Roth,et al.  Joint Aspect and Polarity Classification for Aspect-based Sentiment Analysis with End-to-End Neural Networks , 2018, EMNLP.

[30]  Kathleen M. Carley,et al.  Syntax-Aware Aspect Level Sentiment Classification with Graph Attention Networks , 2019, EMNLP.

[31]  Yue Zhang,et al.  Neural Networks for Open Domain Targeted Sentiment , 2015, EMNLP.

[32]  Luyao Huang,et al.  Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence , 2019, NAACL.

[33]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Erik Cambria,et al.  Targeted Aspect-Based Sentiment Analysis via Embedding Commonsense Knowledge into an Attentive LSTM , 2018, AAAI.

[35]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[36]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[37]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[38]  Mirella Lapata,et al.  Long Short-Term Memory-Networks for Machine Reading , 2016, EMNLP.

[39]  Yujiu Yang,et al.  A Human-Like Semantic Cognition Network for Aspect-Level Sentiment Classification , 2019, AAAI.

[40]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[41]  Eduard H. Hovy,et al.  End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF , 2016, ACL.

[42]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[43]  Chen Zhang,et al.  Aspect-based Sentiment Classification with Aspect-specific Graph Convolutional Networks , 2019, EMNLP/IJCNLP.

[44]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[45]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[46]  Tao Li,et al.  Aspect Based Sentiment Analysis with Gated Convolutional Networks , 2018, ACL.

[47]  Zhen Wu,et al.  Target-oriented Opinion Words Extraction with Target-fused Neural Sequence Labeling , 2019, NAACL.

[48]  Benjamin Van Durme,et al.  Open Domain Targeted Sentiment , 2013, EMNLP.

[49]  Wojciech Zaremba,et al.  An Empirical Exploration of Recurrent Network Architectures , 2015, ICML.

[50]  Sebastian Stabinger,et al.  Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification , 2020, LREC.

[51]  Li Zhao,et al.  Attention-based LSTM for Aspect-level Sentiment Classification , 2016, EMNLP.

[52]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[53]  Guillaume Lample,et al.  Cross-lingual Language Model Pretraining , 2019, NeurIPS.

[54]  Jianfei Yu,et al.  Adapting BERT for Target-Oriented Multimodal Sentiment Classification , 2019, IJCAI.

[55]  Tao Jiang,et al.  Attentional Encoder Network for Targeted Sentiment Classification , 2019, ICANN.

[56]  Dongyan Zhao,et al.  Multi-grained Attention Network for Aspect-Level Sentiment Classification , 2018, EMNLP.

[57]  Hwee Tou Ng,et al.  Exploiting Document Knowledge for Aspect-level Sentiment Classification , 2018, ACL.

[58]  Wei Lu,et al.  Learning Latent Sentiment Scopes for Entity-Level Sentiment Analysis , 2017, AAAI.

[59]  Shuai Wang,et al.  Target-Sensitive Memory Networks for Aspect Sentiment Classification , 2018, ACL.

[60]  Siu Cheung Hui,et al.  Learning to Attend via Word-Aspect Associative Fusion for Aspect-based Sentiment Analysis , 2017, AAAI.

[61]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[62]  Xin Li,et al.  A Unified Model for Opinion Target Extraction and Target Sentiment Prediction , 2018, AAAI.

[63]  Hwee Tou Ng,et al.  An Interactive Multi-Task Learning Network for End-to-End Aspect-Based Sentiment Analysis , 2019, ACL.

[64]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[65]  Xiang Ren,et al.  Empower Sequence Labeling with Task-Aware Neural Language Model , 2017, AAAI.

[66]  Xiaodong Liu,et al.  Unified Language Model Pre-training for Natural Language Understanding and Generation , 2019, NeurIPS.

[67]  Sebastian Ruder,et al.  Universal Language Model Fine-tuning for Text Classification , 2018, ACL.