Memory networks for fine-grained opinion mining

Abstract Fine-grained opinion mining has attracted increasing attention recently because of its benefits for providing richer information compared with coarse-grained sentiment analysis. Under this problem, there are several existing works focusing on aspect (or opinion) terms extraction which utilize the syntactic relations among the words given by a dependency parser. These approaches, however, require additional information and highly depend on the quality of the parsing results. As a result, they may perform poorly on user-generated texts, such as product reviews, tweets, etc., whose syntactic structure is not precise. In this work, we offer an end-to-end deep learning model without any preprocessing. The model consists of a memory network that automatically learns the complicated interactions among aspect words and opinion words. Moreover, we extend the network with a multi-task manner to solve a finer-grained opinion mining problem, which is more challenging than the traditional fine-grained opinion mining problem. To be specific, the finer-grained problem involves identification of aspect and opinion terms within each sentence, as well as categorization of the identified terms at the same time. To this end, we develop an end-to-end multi-task memory network, where aspect/opinion terms extraction for a specific category is considered as a task, and all the tasks are learned jointly by exploring commonalities and relationships among them. We demonstrate state-of-the-art performance of our proposed model on several benchmark datasets.

[1]  Jun Zhao,et al.  Opinion Target Extraction Using Partially-Supervised Word Alignment Model , 2013, IJCAI.

[2]  Hao Yu,et al.  Structure-Aware Review Mining and Summarization , 2010, COLING.

[3]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[4]  Zhong Su,et al.  Product feature categorization with multilevel latent semantic association , 2009, CIKM.

[5]  Hua Xu,et al.  Clustering product features for opinion mining , 2011, WSDM '11.

[6]  Xiaodong Liu,et al.  Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval , 2015, NAACL.

[7]  Xuanjing Huang,et al.  Phrase Dependency Parsing for Opinion Mining , 2009, EMNLP.

[8]  Amit P. Sheth,et al.  Clustering for Simultaneous Extraction of Aspects and Features from Reviews , 2016, NAACL.

[9]  Hwee Tou Ng,et al.  An Unsupervised Neural Attention Model for Aspect Extraction , 2017, ACL.

[10]  Arjun Mukherjee,et al.  Aspect Extraction with Automated Prior Knowledge Learning , 2014, ACL.

[11]  Erik Cambria,et al.  Aspect extraction for opinion mining with a deep convolutional neural network , 2016, Knowl. Based Syst..

[12]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[13]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[14]  Hongfei Yan,et al.  Jointly Modeling Aspects and Opinions with a MaxEnt-LDA Hybrid , 2010, EMNLP.

[15]  Richard Socher,et al.  Ask Me Anything: Dynamic Memory Networks for Natural Language Processing , 2015, ICML.

[16]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[17]  Meng Wang,et al.  Aspect Ranking: Identifying Important Product Aspects from Online Consumer Reviews , 2011, ACL.

[18]  Oren Etzioni,et al.  Extracting Product Features and Opinions from Reviews , 2005, HLT.

[19]  Hua Xu,et al.  Grouping Product Features Using Semi-Supervised Learning with Soft-Constraints , 2010, COLING.

[20]  Alex Graves,et al.  DRAW: A Recurrent Neural Network For Image Generation , 2015, ICML.

[21]  Ivan Titov,et al.  Modeling online reviews with multi-grain topic models , 2008, WWW.

[22]  Jun Zhao,et al.  Opinion Target Extraction Using Word-Based Translation Model , 2012, EMNLP.

[23]  Xiaokui Xiao,et al.  Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis , 2016, EMNLP.

[24]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[25]  Suresh Manandhar,et al.  SemEval-2014 Task 4: Aspect Based Sentiment Analysis , 2014, *SEMEVAL.

[26]  Xin Li,et al.  Deep Multi-Task Learning for Aspect Term Extraction with Memory Interaction , 2017, EMNLP.

[27]  J. Keziya Rani,et al.  Mining Opinion Features in Customer Reviews. , 2016 .

[28]  Xiaokui Xiao,et al.  Coupled Multi-Layer Attentions for Co-Extraction of Aspect and Opinion Terms , 2017, AAAI.

[29]  Phil Blunsom,et al.  Teaching Machines to Read and Comprehend , 2015, NIPS.

[30]  Massimiliano Pontil,et al.  Convex multi-task feature learning , 2008, Machine Learning.

[31]  Suresh Manandhar,et al.  SemEval-2015 Task 12: Aspect Based Sentiment Analysis , 2015, *SEMEVAL.

[32]  Philip S. Yu,et al.  Double Embeddings and CNN-based Sequence Labeling for Aspect Extraction , 2018, ACL.

[33]  Massimiliano Pontil,et al.  Multilinear Multitask Learning , 2013, ICML.

[34]  Jason Weston,et al.  A Neural Attention Model for Abstractive Sentence Summarization , 2015, EMNLP.

[35]  Ivan Titov,et al.  A Joint Model of Text and Aspect Ratings for Sentiment Summarization , 2008, ACL.

[36]  Anton van den Hengel,et al.  Image-Based Recommendations on Styles and Substitutes , 2015, SIGIR.

[37]  Shafiq R. Joty,et al.  Fine-grained Opinion Mining with Recurrent Neural Networks and Word Embeddings , 2015, EMNLP.

[38]  Ming Zhou,et al.  Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction , 2016, IJCAI.

[39]  Yue Lu,et al.  Rated aspect summarization of short comments , 2009, WWW '09.

[40]  Kang Liu,et al.  Book Review: Sentiment Analysis: Mining Opinions, Sentiments, and Emotions by Bing Liu , 2015, CL.

[41]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[42]  Hal Daumé,et al.  Learning Task Grouping and Overlap in Multi-task Learning , 2012, ICML.

[43]  Chun Chen,et al.  Opinion Word Expansion and Target Extraction through Double Propagation , 2011, CL.

[44]  Rich Caruana,et al.  Multitask Learning , 1997, Machine-mediated learning.

[45]  Xinying Xu,et al.  Hidden sentiment association in chinese web opinion mining , 2008, WWW.

[46]  Alex Graves,et al.  Recurrent Models of Visual Attention , 2014, NIPS.

[47]  Jason Weston,et al.  Memory Networks , 2014, ICLR.