Improving Aspect Term Extraction With Bidirectional Dependency Tree Representation

Aspect term extraction is one of the important subtasks in aspect-based sentiment analysis. Previous studies have shown that using dependency tree structure representation is promising for this task. However, most dependency tree structures involve only one directional propagation on the dependency tree. In this paper, we first propose a novel bidirectional dependency tree network to extract dependency structure features from the given sentences. The key idea is to explicitly incorporate both representations gained separately from the bottom-up and top-down propagation on the given dependency syntactic tree. An end-to-end framework is then developed to integrate the embedded representations and BiLSTM plus CRF to learn both tree-structured and sequential features to solve the aspect term extraction problem. Experimental results demonstrate that the proposed model outperforms state-of-the-art baseline models on four benchmark SemEval datasets.

[1]  Chun Chen,et al.  Opinion Word Expansion and Target Extraction through Double Propagation , 2011, CL.

[2]  Eduard H. Hovy,et al.  End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF , 2016, ACL.

[3]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[4]  Xiaokui Xiao,et al.  Coupled Multi-Layer Attentions for Co-Extraction of Aspect and Opinion Terms , 2017, AAAI.

[5]  Liang Lu,et al.  Top-down Tree Long Short-Term Memory Networks , 2015, NAACL.

[6]  Chen Gui,et al.  A Rule-Based Approach to Aspect Extraction from Product Reviews , 2014, SocialNLP@COLING.

[7]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[8]  Erik Cambria,et al.  Aspect extraction for opinion mining with a deep convolutional neural network , 2016, Knowl. Based Syst..

[9]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[10]  Xuanjing Huang,et al.  Phrase Dependency Parsing for Opinion Mining , 2009, EMNLP.

[11]  Christoph Goller,et al.  Learning task-dependent distributed representations by backpropagation through structure , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[12]  Hao Yu,et al.  Structure-Aware Review Mining and Summarization , 2010, COLING.

[13]  Benjamin Van Durme,et al.  Open Domain Targeted Sentiment , 2013, EMNLP.

[14]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[15]  Yue Zhang,et al.  Bidirectional Tree-Structured LSTM with Head Lexicalization , 2016, ArXiv.

[16]  Hwee Tou Ng,et al.  An Unsupervised Neural Attention Model for Aspect Extraction , 2017, ACL.

[17]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[18]  Sinno Jialin Pan,et al.  Multi-task memory networks for category-specific aspect and opinion terms co-extraction , 2017 .

[19]  Jingbo Zhu,et al.  Multi-aspect opinion polling from textual reviews , 2009, CIKM.

[20]  Erik Cambria,et al.  Affective Computing and Sentiment Analysis , 2016, IEEE Intelligent Systems.

[21]  Xiaoyan Zhu,et al.  Movie review mining and summarization , 2006, CIKM '06.

[22]  Andrew Y. Ng,et al.  Parsing Natural Scenes and Natural Language with Recursive Neural Networks , 2011, ICML.

[23]  Jian Su,et al.  NLANGP at SemEval-2016 Task 5: Improving Aspect Based Sentiment Analysis using Neural Network Features , 2016, *SEMEVAL.

[24]  Lei Shu,et al.  Lifelong Learning CRF for Supervised Aspect Extraction , 2017, ACL.

[25]  Yulan He,et al.  Joint sentiment/topic model for sentiment analysis , 2009, CIKM.

[26]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[27]  Oren Etzioni,et al.  Extracting Product Features and Opinions from Reviews , 2005, HLT.

[28]  Iryna Gurevych,et al.  Extracting Opinion Targets in a Single and Cross-Domain Setting with Conditional Random Fields , 2010, EMNLP.

[29]  Thiago Pardo,et al.  NILC_USP: Aspect Extraction using Semantic Labels , 2014, *SEMEVAL.

[30]  Bing Liu,et al.  Mining Aspect-Specific Opinion using a Holistic Lifelong Topic Model , 2016, WWW.

[31]  Martin Ester,et al.  ILDA: interdependent LDA model for learning latent aspects and their ratings from online product reviews , 2011, SIGIR.

[32]  Arjun Mukherjee,et al.  Exploiting Domain Knowledge in Aspect Extraction , 2013, EMNLP.

[33]  Xin Li,et al.  Deep Multi-Task Learning for Aspect Term Extraction with Memory Interaction , 2017, EMNLP.

[34]  Bing Liu,et al.  Topic Modeling using Topics from Many Domains, Lifelong Learning and Big Data , 2014, ICML.

[35]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[36]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[37]  Suk Hwan Lim,et al.  Extracting and Ranking Product Features in Opinion Documents , 2010, COLING.

[38]  Zhiqiang Toh,et al.  DLIREC: Aspect Term Extraction and Term Polarity Classification System , 2014, *SEMEVAL.

[39]  Haris Papageorgiou,et al.  SemEval-2016 Task 5: Aspect Based Sentiment Analysis , 2016, *SEMEVAL.

[40]  Hai Ye,et al.  Dependency-Tree Based Convolutional Neural Networks for Aspect Term Extraction , 2017, PAKDD.

[41]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[42]  Wei Xu,et al.  Bidirectional LSTM-CRF Models for Sequence Tagging , 2015, ArXiv.

[43]  Lidong Bing,et al.  Recurrent Attention Network on Memory for Aspect Sentiment Analysis , 2017, EMNLP.

[44]  Rich Caruana,et al.  Overfitting in Neural Nets: Backpropagation, Conjugate Gradient, and Early Stopping , 2000, NIPS.

[45]  Maryna Chernyshevich,et al.  IHS R&D Belarus: Cross-domain extraction of product features using CRF , 2014, *SEMEVAL.

[46]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[47]  Rodrigo Agerri,et al.  EliXa: A Modular and Flexible ABSA Platform , 2015, *SEMEVAL.

[48]  Jian Su,et al.  NLANGP: Supervised Machine Learning System for Aspect Category Classification and Opinion Target Extraction , 2015, *SEMEVAL.

[49]  Xiaokui Xiao,et al.  Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis , 2016, EMNLP.

[50]  Qian Liu,et al.  A Logic Programming Approach to Aspect Extraction in Opinion Mining , 2013, 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT).

[51]  Suresh Manandhar,et al.  SemEval-2014 Task 4: Aspect Based Sentiment Analysis , 2014, *SEMEVAL.

[52]  Frédéric Béchet,et al.  Lsislif: CRF and Logistic Regression for Opinion Target Extraction and Sentiment Polarity Analysis , 2015, SemEval@NAACL-HLT.

[53]  Shujian Huang,et al.  Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder , 2017, ACL.

[54]  Noémie Elhadad,et al.  An Unsupervised Aspect-Sentiment Model for Online Reviews , 2010, NAACL.

[55]  Claire Cardie,et al.  Opinion Mining with Deep Recurrent Neural Networks , 2014, EMNLP.

[56]  Sinno Jialin Pan,et al.  Multi-task Coupled Attentions for Category-specific Aspect and Opinion Terms Co-extraction , 2017, ArXiv.

[57]  Bo Wang,et al.  Bootstrapping Both Product Features and Opinion Words from Chinese Customer Reviews with Cross-Inducing , 2008, IJCNLP.

[58]  Makoto Miwa,et al.  End-to-End Relation Extraction using LSTMs on Sequences and Tree Structures , 2016, ACL.

[59]  Shafiq R. Joty,et al.  Fine-grained Opinion Mining with Recurrent Neural Networks and Word Embeddings , 2015, EMNLP.

[60]  Ming Zhou,et al.  Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction , 2016, IJCAI.

[61]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[62]  Claire Cardie,et al.  Hierarchical Sequential Learning for Extracting Opinions and Their Attributes , 2010, ACL.

[63]  Qian Liu,et al.  Improving Opinion Aspect Extraction Using Semantic Similarity and Aspect Associations , 2016, AAAI.

[64]  Claudiu Musat,et al.  Unsupervised Aspect Term Extraction with B-LSTM & CRF using Automatically Labelled Datasets , 2017, WASSA@EMNLP.

[65]  Heng Ji,et al.  A Dependency-Based Neural Network for Relation Classification , 2015, ACL.

[66]  Arjun Mukherjee,et al.  Aspect Extraction with Automated Prior Knowledge Learning , 2014, ACL.

[67]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[68]  Qian Liu,et al.  Automated Rule Selection for Aspect Extraction in Opinion Mining , 2015, IJCAI.

[69]  Claire Cardie,et al.  Bidirectional Recursive Neural Networks for Token-Level Labeling with Structure , 2013, ArXiv.