A Bayesian Model for Joint Unsupervised Induction of Sentiment, Aspect and Discourse Representations

We propose a joint model for unsupervised induction of sentiment, aspect and discourse information and show that by incorporating a notion of latent discourse relations in the model, we improve the prediction accuracy for aspect and sentiment polarity on the sub-sentential level. We deviate from the traditional view of discourse, as we induce types of discourse relations and associated discourse cues relevant to the considered opinion analysis task; consequently, the induced discourse relations play the role of opinion and aspect shifters. The quantitative analysis that we conducted indicated that the integration of a discourse model increased the prediction accuracy results with respect to the discourse-agnostic approach and the qualitative analysis suggests that the induced representations encode a meaningful discourse structure.

[1]  Ivan Titov,et al.  Modeling online reviews with multi-grain topic models , 2008, WWW.

[2]  Kentaro Inui,et al.  Dependency Tree-based Sentiment Classification using CRFs with Hidden Variables , 2010, NAACL.

[3]  Swapna Somasundaran,et al.  Discourse Level Opinion Interpretation , 2008, COLING.

[4]  Jeffrey Pennington,et al.  Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions , 2011, EMNLP.

[5]  Hal Daumé,et al.  Incorporating Lexical Priors into Topic Models , 2012, EACL.

[6]  Annie Zaenen,et al.  Contextual Valence Shifters , 2006, Computing Attitude and Affect in Text.

[7]  Regina Barzilay,et al.  Multiple Aspect Ranking Using the Good Grief Algorithm , 2007, NAACL.

[8]  Regina Barzilay,et al.  Bayesian Unsupervised Topic Segmentation , 2008, EMNLP.

[9]  Mark Steyvers,et al.  Finding scientific topics , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[10]  TaboadaMaite,et al.  Lexicon-based methods for sentiment analysis , 2011 .

[11]  Jacob Eisenstein,et al.  Discourse Connectors for Latent Subjectivity in Sentiment Analysis , 2013, NAACL.

[12]  Oren Etzioni,et al.  Extracting Product Features and Opinions from Reviews , 2005, HLT.

[13]  Kimberly D. Voll,et al.  Extracting sentiment as a function of discourse structure and topicality , 2008 .

[14]  Eneko Agirre,et al.  Semeval-2007 Task 2 : Evaluating Word Sense Induction and Discrimination , 2007 .

[15]  T. Griffiths,et al.  A Bayesian framework for word segmentation: Exploring the effects of context , 2009, Cognition.

[16]  Regina Barzilay,et al.  Multilingual Part-of-Speech Tagging: Two Unsupervised Approaches , 2009, J. Artif. Intell. Res..

[17]  Xu Ling,et al.  Topic sentiment mixture: modeling facets and opinions in weblogs , 2007, WWW '07.

[18]  Michael L. Littman,et al.  Unsupervised Learning of Semantic Orientation from a Hundred-Billion-Word Corpus , 2002, ArXiv.

[19]  Ivan Titov,et al.  A Joint Model of Text and Aspect Ratings for Sentiment Summarization , 2008, ACL.

[20]  Alice H. Oh,et al.  Aspect and sentiment unification model for online review analysis , 2011, WSDM '11.

[21]  Hongfei Yan,et al.  Jointly Modeling Aspects and Opinions with a MaxEnt-LDA Hybrid , 2010, EMNLP.

[22]  Maite Taboada,et al.  Not All Words Are Created Equal: Extracting Semantic Orientation as a Function of Adjective Relevance , 2007, Australian Conference on Artificial Intelligence.

[23]  Noémie Elhadad,et al.  An Unsupervised Aspect-Sentiment Model for Online Reviews , 2010, NAACL.

[24]  Max Welling Donald,et al.  Products of Experts , 2007 .

[25]  Maite Taboada,et al.  A Syntactic and Lexical-Based Discourse Segmenter , 2009, ACL.

[26]  Oscar Täckström,et al.  Semi-supervised latent variable models for sentence-level sentiment analysis , 2011, ACL.

[27]  Nicholas Asher,et al.  Distilling Opinion in Discourse: A Preliminary Study , 2008, COLING.

[28]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[29]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[30]  Kalina Bontcheva,et al.  Human Language Technologies , 2009, Semantic Knowledge Management.

[31]  Michael I. Jordan,et al.  An HDP-HMM for systems with state persistence , 2008, ICML '08.

[32]  Yulan He,et al.  Joint sentiment/topic model for sentiment analysis , 2009, CIKM.

[33]  Wei Gao,et al.  Unsupervised Discovery of Discourse Relations for Eliminating Intra-sentence Polarity Ambiguities , 2011, EMNLP.

[34]  Livio Robaldo,et al.  The Penn Discourse TreeBank 2.0. , 2008, LREC.

[35]  Daniel Marcu,et al.  Sentence Level Discourse Parsing using Syntactic and Lexical Information , 2003, NAACL.

[36]  Bonnie L. Webber,et al.  Discourse structure and language technology , 2011, Natural Language Engineering.

[37]  William C. Mann,et al.  Rhetorical Structure Theory: Toward a functional theory of text organization , 1988 .

[38]  Amélie Marian,et al.  Beyond the Stars: Improving Rating Predictions using Review Text Content , 2009, WebDB.

[39]  Lise Getoor,et al.  Supervised and Unsupervised Methods in Employing Discourse Relations for Improving Opinion Polarity Classification , 2009, EMNLP.

[40]  Mikio Yamamoto,et al.  Sentiment Analysis Based on Probabilistic Models Using Inter-Sentence Information , 2008, LREC.