Aspect opinion expression and rating prediction via LDA–CRF hybrid

In this paper, we study the problem of aspect-based sentiment analysis. Our model simultaneously extracts aspect-specific opinion expressions and determines the rating for each aspect in reviews. Previous works have mainly focused on the problem of opinion phrase extraction and aspect rating prediction in a pipelined manner and are not able to capture the dependencies of aspect opinion expression on aspect rating and vice-versa. They are also unable to discover aspect-specific opinion expressions and their associated rating scores. We present a joint modelling approach to extract aspect-specific sentiment expression and aspect rating prediction simultaneously. This paper proposes a novel LDA–CRF hybrid model which employs discriminative conditional random field component for phrase extraction, a regression component for rating prediction and a generative component for grouping aspect–sentiment expressions (aspect-specific opinion expressions) into coherent topics. To show the effectiveness of our approach, we evaluate the performance of the model on both task: (i) aspect-specific opinion expressions and (ii) rating prediction on the dataset of hotel and restaurant reviews from TripAdvisor.com. Experimental results show that both task potentially reinforce each other and joint modeling outperformed state-of-the-art baselines for each individual tasks.

[1]  Yue Lu,et al.  Rated aspect summarization of short comments , 2009, WWW '09.

[2]  Xuanjing Huang,et al.  Phrase Dependency Parsing for Opinion Mining , 2009, EMNLP.

[3]  Clare R. Voss,et al.  Scalable Topical Phrase Mining from Text Corpora , 2014, Proc. VLDB Endow..

[4]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[5]  Hosam M. Mahmoud,et al.  Polya Urn Models , 2008 .

[6]  Claire Cardie,et al.  Opinion Mining with Deep Recurrent Neural Networks , 2014, EMNLP.

[7]  Arjun Mukherjee,et al.  Aspect Extraction through Semi-Supervised Modeling , 2012, ACL.

[8]  Claire Cardie,et al.  Identifying Expressions of Opinion in Context , 2007, IJCAI.

[9]  Ivan Titov,et al.  A Joint Model of Text and Aspect Ratings for Sentiment Summarization , 2008, ACL.

[10]  Yang Song,et al.  Topical Keyphrase Extraction from Twitter , 2011, ACL.

[11]  Claire Cardie,et al.  Joint Inference for Fine-grained Opinion Extraction , 2013, ACL.

[12]  Shafiq R. Joty,et al.  Fine-grained Opinion Mining with Recurrent Neural Networks and Word Embeddings , 2015, EMNLP.

[13]  Ming Zhou,et al.  Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction , 2016, IJCAI.

[14]  Regina Barzilay,et al.  Content Models with Attitude , 2011, ACL.

[15]  Xiaoyan Zhu,et al.  Movie review mining and summarization , 2006, CIKM '06.

[16]  Claire Cardie,et al.  Annotating Expressions of Opinions and Emotions in Language , 2005, Lang. Resour. Evaluation.

[17]  Kun Yang,et al.  Dynamic non-parametric joint sentiment topic mixture model , 2015, Knowl. Based Syst..

[18]  Andrew McCallum,et al.  Optimizing Semantic Coherence in Topic Models , 2011, EMNLP.

[19]  Lei Zhang,et al.  A Survey of Opinion Mining and Sentiment Analysis , 2012, Mining Text Data.

[20]  Claire Cardie,et al.  Joint Modeling of Opinion Expression Extraction and Attribute Classification , 2014, Transactions of the Association for Computational Linguistics.

[21]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[22]  Yue Lu,et al.  Latent aspect rating analysis without aspect keyword supervision , 2011, KDD.

[23]  Xu Ling,et al.  Topic sentiment mixture: modeling facets and opinions in weblogs , 2007, WWW '07.

[24]  Claire Cardie,et al.  Extracting Opinion Expressions with semi-Markov Conditional Random Fields , 2012, EMNLP.

[25]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[26]  Yulan He,et al.  Joint sentiment/topic model for sentiment analysis , 2009, CIKM.

[27]  David M. Pennock,et al.  Mining the peanut gallery: opinion extraction and semantic classification of product reviews , 2003, WWW '03.

[28]  Samy Bengio,et al.  Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks , 2015, NIPS.

[29]  Xianghua Fu,et al.  Multi-aspect sentiment analysis for Chinese online social reviews based on topic modeling and HowNet lexicon , 2013, Knowl. Based Syst..

[30]  Eduard Hovy,et al.  Extracting Opinions, Opinion Holders, and Topics Expressed in Online News Media Text , 2006 .

[31]  Xiaoyan Zhu,et al.  Sentiment Analysis with Global Topics and Local Dependency , 2010, AAAI.

[32]  Kiyoaki Shirai,et al.  PhraseRNN: Phrase Recursive Neural Network for Aspect-based Sentiment Analysis , 2015, EMNLP.

[33]  Lise Getoor,et al.  Supervised and Unsupervised Methods in Employing Discourse Relations for Improving Opinion Polarity Classification , 2009, EMNLP.

[34]  Haris Papageorgiou,et al.  SemEval-2016 Task 5: Aspect Based Sentiment Analysis , 2016, *SEMEVAL.

[35]  Wei Xu,et al.  Bidirectional LSTM-CRF Models for Sequence Tagging , 2015, ArXiv.

[36]  Andrew McCallum,et al.  Topic Models Conditioned on Arbitrary Features with Dirichlet-multinomial Regression , 2008, UAI.

[37]  Erik Cambria,et al.  Sentic LDA: Improving on LDA with semantic similarity for aspect-based sentiment analysis , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[38]  Yuji Matsumoto,et al.  Extracting Aspect-Evaluation and Aspect-Of Relations in Opinion Mining , 2007, EMNLP.

[39]  Martin Ester,et al.  The FLDA model for aspect-based opinion mining: addressing the cold start problem , 2013, WWW.

[40]  Richard Socher,et al.  Aspect Specific Sentiment Analysis Using Hierarchical Deep Learning , 2014 .

[41]  Claire Cardie,et al.  Joint Extraction of Entities and Relations for Opinion Recognition , 2006, EMNLP.

[42]  Hao Wang,et al.  A Sentiment-aligned Topic Model for Product Aspect Rating Prediction , 2014, EMNLP.

[43]  Richard Johansson,et al.  Extracting Opinion Expressions and Their Polarities - Exploration of Pipelines and Joint Models , 2011, ACL.

[44]  Li Zhao,et al.  Attention-based LSTM for Aspect-level Sentiment Classification , 2016, EMNLP.

[45]  Claire Cardie,et al.  Identifying Sources of Opinions with Conditional Random Fields and Extraction Patterns , 2005, HLT.

[46]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[47]  Oren Etzioni,et al.  Extracting Product Features and Opinions from Reviews , 2005, HLT.

[48]  Jiawei Han,et al.  Mining Quality Phrases from Massive Text Corpora , 2015, SIGMOD Conference.

[49]  A. McCallum,et al.  Topical N-Grams: Phrase and Topic Discovery, with an Application to Information Retrieval , 2007, Seventh IEEE International Conference on Data Mining (ICDM 2007).

[50]  Noémie Elhadad,et al.  An Unsupervised Aspect-Sentiment Model for Online Reviews , 2010, NAACL.

[51]  Wai Lam,et al.  Latent Aspect Mining via Exploring Sparsity and Intrinsic Information , 2014, CIKM.

[52]  Eric P. Xing,et al.  MedLDA: maximum margin supervised topic models , 2012, J. Mach. Learn. Res..

[53]  Robert V. Lindsey,et al.  A Phrase-Discovering Topic Model Using Hierarchical Pitman-Yor Processes , 2012, EMNLP.

[54]  David M. Blei,et al.  Supervised Topic Models , 2007, NIPS.

[55]  Himabindu Lakkaraju,et al.  Exploiting Coherence for the Simultaneous Discovery of Latent Facets and associated Sentiments , 2011, SDM.

[56]  Iryna Gurevych,et al.  Extracting Opinion Targets in a Single and Cross-Domain Setting with Conditional Random Fields , 2010, EMNLP.

[57]  Claire Cardie,et al.  Multi-aspect Sentiment Analysis with Topic Models , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[58]  Xiaocheng Feng,et al.  Effective LSTMs for Target-Dependent Sentiment Classification , 2015, COLING.

[59]  Regina Barzilay,et al.  Automatic Aggregation by Joint Modeling of Aspects and Values , 2014, J. Artif. Intell. Res..

[60]  Erik Cambria,et al.  Aspect extraction for opinion mining with a deep convolutional neural network , 2016, Knowl. Based Syst..

[61]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[62]  Alice H. Oh,et al.  Aspect and sentiment unification model for online review analysis , 2011, WSDM '11.

[63]  Hongfei Yan,et al.  Jointly Modeling Aspects and Opinions with a MaxEnt-LDA Hybrid , 2010, EMNLP.

[64]  Laizhong Cui,et al.  Topic Sentiment Joint Model with Word Embeddings , 2016, DMNLP@PKDD/ECML.

[65]  Yue Lu,et al.  Latent aspect rating analysis on review text data: a rating regression approach , 2010, KDD.

[66]  Regina Barzilay,et al.  Multiple Aspect Ranking Using the Good Grief Algorithm , 2007, NAACL.

[67]  Alexander J. Smola,et al.  Jointly modeling aspects, ratings and sentiments for movie recommendation (JMARS) , 2014, KDD.

[68]  Martin Ester,et al.  ILDA: interdependent LDA model for learning latent aspects and their ratings from online product reviews , 2011, SIGIR.