Leveraging Structural and Semantic Correspondence for Attribute-Oriented Aspect Sentiment Discovery

Opinionated text often involves attributes such as authorship and location that influence the sentiments expressed for different aspects. We posit that structural and semantic correspondence is both prevalent in opinionated text, especially when associated with attributes, and crucial in accurately revealing its latent aspect and sentiment structure. However, it is not recognized by existing approaches. We propose Trait, an unsupervised probabilistic model that discovers aspects and sentiments from text and associates them with different attributes. To this end, Trait infers and leverages structural and semantic correspondence using a Markov Random Field. We show empirically that by incorporating attributes explicitly Trait significantly outperforms state-of-the-art baselines both by generating attribute profiles that accord with our intuitions, as shown via visualization, and yielding topics of greater semantic cohesion.

[1]  Ivan Titov,et al.  Modeling online reviews with multi-grain topic models , 2008, WWW.

[2]  Philip Resnik,et al.  Adapting Topic Models using Lexical Associations with Tree Priors , 2017, EMNLP.

[3]  Lili Mou,et al.  Disentangled Representation Learning for Non-Parallel Text Style Transfer , 2018, ACL.

[4]  Ivan Titov,et al.  A Joint Model of Text and Aspect Ratings for Sentiment Summarization , 2008, ACL.

[5]  Christian S. Perone,et al.  Evaluation of sentence embeddings in downstream and linguistic probing tasks , 2018, ArXiv.

[6]  B. MacWhinney A UNIFIED MODEL , 2007 .

[7]  A. Baharuddin,et al.  The impact of geographical location on taste sensitivity and preference. , 2015 .

[8]  Stefan M. Rüger,et al.  Weakly Supervised Joint Sentiment-Topic Detection from Text , 2012, IEEE Transactions on Knowledge and Data Engineering.

[9]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[10]  Zhe Zhang,et al.  Limbic: Author-Based Sentiment Aspect Modeling Regularized with Word Embeddings and Discourse Relations , 2018, EMNLP.

[11]  Mong-Li Lee,et al.  Author-aware Aspect Topic Sentiment Model to Retrieve Supporting Opinions from Reviews , 2017, EMNLP.

[12]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[13]  Subhabrata Mukherjee,et al.  Joint Author Sentiment Topic Model , 2014, SDM.

[14]  Jun S. Liu,et al.  The Collapsed Gibbs Sampler in Bayesian Computations with Applications to a Gene Regulation Problem , 1994 .

[15]  Josh H. McDermott,et al.  Indifference to dissonance in native Amazonians reveals cultural variation in music perception , 2016, Nature.

[16]  Thang Nguyen,et al.  Is Your Anchor Going Up or Down? Fast and Accurate Supervised Topic Models , 2015, NAACL.

[17]  Andrew McCallum,et al.  Optimizing Semantic Coherence in Topic Models , 2011, EMNLP.

[18]  Thomas L. Griffiths,et al.  The Author-Topic Model for Authors and Documents , 2004, UAI.

[19]  Bing Liu,et al.  Mining Aspect-Specific Opinion using a Holistic Lifelong Topic Model , 2016, WWW.

[20]  Timothy Baldwin,et al.  Machine Reading Tea Leaves: Automatically Evaluating Topic Coherence and Topic Model Quality , 2014, EACL.

[21]  Lei Li,et al.  Generating Sentences from Disentangled Syntactic and Semantic Spaces , 2019, ACL.

[22]  Nan Hua,et al.  Universal Sentence Encoder for English , 2018, EMNLP.

[23]  Byron C. Wallace,et al.  Learning Disentangled Representations of Texts with Application to Biomedical Abstracts , 2018, EMNLP.

[24]  Arjun Mukherjee,et al.  Exploiting Domain Knowledge in Aspect Extraction , 2013, EMNLP.

[25]  Alice H. Oh,et al.  A Hierarchical Aspect-Sentiment Model for Online Reviews , 2013, AAAI.

[26]  Hosam M. Mahmoud,et al.  Polya Urn Models , 2008 .

[27]  Dat Quoc Nguyen,et al.  Improving Topic Models with Latent Feature Word Representations , 2015, TACL.

[28]  Bing Liu,et al.  Review Topic Discovery with Phrases using the Pólya Urn Model , 2014, COLING.

[29]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[30]  Christopher D. Manning,et al.  Incorporating Non-local Information into Information Extraction Systems by Gibbs Sampling , 2005, ACL.

[31]  Dominik Endres,et al.  A new metric for probability distributions , 2003, IEEE Transactions on Information Theory.

[32]  Derek Greene,et al.  An analysis of the coherence of descriptors in topic modeling , 2015, Expert Syst. Appl..

[33]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[34]  Michael L. Littman,et al.  Measuring praise and criticism: Inference of semantic orientation from association , 2003, TOIS.

[35]  Alice H. Oh,et al.  Aspect and sentiment unification model for online review analysis , 2011, WSDM '11.

[36]  Jing Jiang,et al.  A Unified Model for Topics, Events and Users on Twitter , 2013, EMNLP.

[37]  Yizhou Sun,et al.  ETM: Entity Topic Models for Mining Documents Associated with Entities , 2012, 2012 IEEE 12th International Conference on Data Mining.

[38]  Kevin Gimpel,et al.  A Multi-Task Approach for Disentangling Syntax and Semantics in Sentence Representations , 2019, NAACL.