A Vector Space Approach for Aspect Based Sentiment Analysis

Vector representations for language has been shown to be useful in a number of Natural Language Processing tasks. In this paper, we aim to investigate the effectiveness of word vector representations for the problem of Aspect Based Sentiment Analysis. In particular, we target three sub-tasks namely aspect term extraction, aspect category detection, and aspect sentiment prediction. We investigate the effectiveness of vector representations over different text data and evaluate the quality of domain-dependent vectors. We utilize vector representations to compute various vectorbased features and conduct extensive experiments to demonstrate their effectiveness. Using simple vector based features, we achieve F1 scores of 79.91% for aspect term extraction, 86.75% for category detection, and the accuracy 72.39% for aspect sentiment prediction.

[1]  Jordan L. Boyd-Graber,et al.  Besting the Quiz Master: Crowdsourcing Incremental Classification Games , 2012, EMNLP.

[2]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[3]  Claire Cardie,et al.  Multi-aspect Sentiment Analysis with Topic Models , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[4]  Eric K. Ringger,et al.  Pulse: Mining Customer Opinions from Free Text , 2005, IDA.

[5]  Christopher Potts,et al.  Learning the meaning of scalar adjectives , 2010 .

[6]  Christopher D. Manning,et al.  The Stanford Typed Dependencies Representation , 2008, CF+CDPE@COLING.

[7]  Phil Blunsom,et al.  Semantic Role Labelling with Tree Conditional Random Fields , 2005, CoNLL.

[8]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.

[9]  Noémie Elhadad,et al.  An Unsupervised Aspect-Sentiment Model for Online Reviews , 2010, NAACL.

[10]  Alice H. Oh,et al.  Aspect and sentiment unification model for online review analysis , 2011, WSDM '11.

[11]  Brendan T. O'Connor,et al.  From Tweets to Polls: Linking Text Sentiment to Public Opinion Time Series , 2010, ICWSM.

[12]  Geoffrey Zweig,et al.  Linguistic Regularities in Continuous Space Word Representations , 2013, NAACL.

[13]  Hongfei Yan,et al.  Jointly Modeling Aspects and Opinions with a MaxEnt-LDA Hybrid , 2010, EMNLP.

[14]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2011, Math. Program..

[15]  Yulan He,et al.  Joint sentiment/topic model for sentiment analysis , 2009, CIKM.

[16]  Claire Cardie,et al.  Joint Inference for Fine-grained Opinion Extraction , 2013, ACL.

[17]  Kevin Gimpel,et al.  Tailoring Continuous Word Representations for Dependency Parsing , 2014, ACL.

[18]  J. R. Firth,et al.  A Synopsis of Linguistic Theory, 1930-1955 , 1957 .

[19]  Richard Socher,et al.  A Neural Network for Factoid Question Answering over Paragraphs , 2014, EMNLP.

[20]  Steven Skiena,et al.  Polyglot: Distributed Word Representations for Multilingual NLP , 2013, CoNLL.

[21]  Steven Bethard,et al.  Building Test Suites for UIMA Components , 2009 .

[22]  Caroline Brun,et al.  XRCE: Hybrid Classification for Aspect-based Sentiment Analysis , 2014, *SEMEVAL.

[23]  H. Schütze,et al.  Dimensions of meaning , 1992, Supercomputing '92.

[24]  Christopher S. G. Khoo,et al.  Aspect-based sentiment analysis of movie reviews on discussion boards , 2010, J. Inf. Sci..

[25]  Xiaojun Wan,et al.  Collective Opinion Target Extraction in Chinese Microblogs , 2013, EMNLP.

[26]  Suresh Manandhar,et al.  SemEval-2014 Task 4: Aspect Based Sentiment Analysis , 2014, *SEMEVAL.

[27]  Tat-Seng Chua,et al.  Mining slang and urban opinion words and phrases from cQA services: an optimization approach , 2012, WSDM '12.

[28]  Xu Ling,et al.  Topic sentiment mixture: modeling facets and opinions in weblogs , 2007, WWW '07.

[29]  Pavel Blinov,et al.  Blinov: Distributed Representations of Words for Aspect-Based Sentiment Analysis at SemEval 2014 , 2014, *SEMEVAL.

[30]  Geoffrey Zweig,et al.  Joint semantic utterance classification and slot filling with recursive neural networks , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[31]  Omer Levy,et al.  Dependency-Based Word Embeddings , 2014, ACL.

[32]  Christopher Potts,et al.  Recursive Neural Networks for Learning Logical Semantics , 2014, ArXiv.

[33]  J. Nocedal Updating Quasi-Newton Matrices With Limited Storage , 1980 .

[34]  Thomas Hofmann,et al.  Hidden Markov Support Vector Machines , 2003, ICML.

[35]  Steven Skiena,et al.  Trading Strategies to Exploit Blog and News Sentiment , 2010, ICWSM.

[36]  Lyric Doshi,et al.  Using sentiment and social network analyses to predict opening-movie box-office success , 2010 .

[37]  Maryna Chernyshevich,et al.  IHS R&D Belarus: Cross-domain Extraction of Product Features using Conditional Random Fields , 2014 .

[38]  Oliver Ferschke,et al.  DKPro TC: A Java-based Framework for Supervised Learning Experiments on Textual Data , 2014, ACL.

[39]  Vysoké Učení,et al.  Statistical Language Models Based on Neural Networks , 2012 .

[40]  Hinrich Schütze,et al.  Word Space , 1992, NIPS.

[41]  Jun Zhao,et al.  Mining Opinion Words and Opinion Targets in a Two-Stage Framework , 2013, ACL.

[42]  Andrew McCallum,et al.  Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data , 2001, ICML.

[43]  Zhiqiang Toh,et al.  DLIREC: Aspect Term Extraction and Term Polarity Classification System , 2014, *SEMEVAL.

[44]  Xiaohui Yu,et al.  ARSA: a sentiment-aware model for predicting sales performance using blogs , 2007, SIGIR.

[45]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[46]  Joachim Wagner,et al.  DCU: Aspect-based Polarity Classification for SemEval Task 4 , 2014, *SEMEVAL.

[47]  Alexander I. Rudnicky,et al.  Dynamically supporting unexplored domains in conversational interactions by enriching semantics with neural word embeddings , 2014, 2014 IEEE Spoken Language Technology Workshop (SLT).

[48]  Oren Etzioni,et al.  Extracting Product Features and Opinions from Reviews , 2005, HLT.

[49]  Navneet Kaur,et al.  Opinion mining and sentiment analysis , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[50]  Saif Mohammad,et al.  NRC-Canada-2014: Recent Improvements in the Sentiment Analysis of Tweets , 2014, SemEval@COLING.

[51]  Claire Cardie,et al.  Hierarchical Sequential Learning for Extracting Opinions and Their Attributes , 2010, ACL.

[52]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[53]  Thorsten Joachims,et al.  Cutting-plane training of structural SVMs , 2009, Machine Learning.

[54]  Frank Keller,et al.  Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, June 22-27, 2014, Baltimore, MD, USA, Volume 1: Long Papers , 2014, ACL.

[55]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[56]  Philipp Koehn,et al.  Synthesis Lectures on Human Language Technologies , 2016 .

[57]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[58]  Ting Liu,et al.  Collocation Polarity Disambiguation Using Web-based Pseudo Contexts , 2012, EMNLP.

[59]  Andrea Esuli,et al.  Multi-Faceted Rating of Product Reviews , 2009, ERCIM News.

[60]  Steven Skiena,et al.  The Wisdom of Bookies? Sentiment Analysis Versus. the NFL Point Spread , 2010, ICWSM.

[61]  Johan Bollen,et al.  Twitter mood predicts the stock market , 2010, J. Comput. Sci..

[62]  Fernando Pereira,et al.  Shallow Parsing with Conditional Random Fields , 2003, NAACL.

[63]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[64]  Kenneth Ward Church,et al.  Word2Vec , 2016, Natural Language Engineering.

[65]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[66]  Chong Long,et al.  A Review Selection Approach for Accurate Feature Rating Estimation , 2010, COLING.

[67]  Iryna Gurevych,et al.  A broad-coverage collection of portable NLP components for building shareable analysis pipelines , 2014, OIAF4HLT@COLING.

[68]  Amélie Marian,et al.  Beyond the Stars: Improving Rating Predictions using Review Text Content , 2009, WebDB.

[69]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[70]  Koby Crammer,et al.  Confidence in Structured-Prediction Using Confidence-Weighted Models , 2010, EMNLP.

[71]  Braja Gopal Patra,et al.  JU_CSE: A Conditional Random Field (CRF) Based Approach to Aspect Based Sentiment Analysis , 2014, *SEMEVAL.

[72]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[73]  Maria Leonor Pacheco,et al.  of the Association for Computational Linguistics: , 2001 .

[74]  Peter D. Turney Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews , 2002, ACL.