Two-stage sentiment classification based on user-product interactive information

Abstract Document-level review sentiment classification aims to predict the sentiment category for given review documents written by users for products. Most of the existing methods focus on generating a good review document representation and classifying the review document directly. However, on the one hand, as document-level review sentiment classification usually includes many sentiment categories and the difference between these sentiment categories is not obvious, it may be difficult to obtain satisfying result by direct classification. On the other hand, this once classification process with review representation may fail to well interpret how the results are achieved. In addition, although some information such as user preference and product characteristics are incorporated when building models, the interactive information between user and product are usually ignored. In this paper, inspired by the deductive reasoning strategy of human doing multiple choice questions, we are motivated to propose a Two-Stage Sentiment Classification (TSSC) model to classify review documents in two stages: (1) Coarse classification stage, where model mainly adopts user-product interactive information to pre-judge the sentiment tendency of the review document without considering the review information; (2) Fine classification stage, where model uses text information of the review document for further classification based on the sentiment tendency obtained in coarse classification stage. Finally, the sentiment classification task is accomplished by combining both the results of coarse classification and fine classification. The experimental results demonstrate that our TSSC model significantly outperforms most of the related models (e.g., Trigram and NSC+UPA) on IMDB and Yelp datasets in terms of classification accuracy. When compared with the state-of-the-art HUAPA model, our TSSC model not only achieves slightly more accurate performance, but also has lower time complexity and stronger interpretability.

[1]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[2]  Andrew Y. Ng,et al.  Parsing Natural Scenes and Natural Language with Recursive Neural Networks , 2011, ICML.

[3]  Navneet Kaur,et al.  Opinion mining and sentiment analysis , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[4]  Yoshua Bengio,et al.  Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach , 2011, ICML.

[5]  Jeffrey Pennington,et al.  Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions , 2011, EMNLP.

[6]  J. Madaus,et al.  The Test‐Taking Strategy Intervention for College Students with Learning Disabilities , 2009 .

[7]  Don Steinberg,et al.  Deductive reasoning , 1989 .

[8]  Tao Chen,et al.  Learning User and Product Distributed Representations Using a Sequence Model for Sentiment Analysis , 2016, IEEE Computational Intelligence Magazine.

[9]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[10]  Hongyu Guo,et al.  Long Short-Term Memory Over Recursive Structures , 2015, ICML.

[11]  Zhiyuan Liu,et al.  Neural Sentiment Classification with User and Product Attention , 2016, EMNLP.

[12]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[13]  Ting Liu,et al.  Document Modeling with Gated Recurrent Neural Network for Sentiment Classification , 2015, EMNLP.

[14]  Fangzhao Wu,et al.  Domain attention model for multi-domain sentiment classification , 2018, Knowl. Based Syst..

[15]  M. N. Sulaiman,et al.  A Review On Evaluation Metrics For Data Classification Evaluations , 2015 .

[16]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[17]  Tong Zhang,et al.  Effective Use of Word Order for Text Categorization with Convolutional Neural Networks , 2014, NAACL.

[18]  Masaru Kitsuregawa,et al.  Modeling User Leniency and Product Popularity for Sentiment Classification , 2013, IJCNLP.

[19]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[20]  Andrew Y. Ng,et al.  Semantic Compositionality through Recursive Matrix-Vector Spaces , 2012, EMNLP.

[21]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[22]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[23]  Gautam Das,et al.  A Human-in-the-loop Attribute Design Framework for Classification , 2019, WWW.

[24]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[25]  Randolph E. Sarnacki,et al.  An Examination of Test-Wiseness In the Cognitive Test Domain , 1979 .

[26]  Xuanjing Huang,et al.  Cached Long Short-Term Memory Neural Networks for Document-Level Sentiment Classification , 2016, EMNLP.

[27]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[28]  Saif Mohammad,et al.  Sentiment Analysis of Short Informal Texts , 2014, J. Artif. Intell. Res..

[29]  Yehuda Koren,et al.  Advances in Collaborative Filtering , 2011, Recommender Systems Handbook.

[30]  Ting Liu,et al.  Learning Semantic Representations of Users and Products for Document Level Sentiment Classification , 2015, ACL.

[31]  P N Johnson-Laird,et al.  Deductive reasoning. , 1999, Annual review of psychology.

[32]  Gerhard Weikum,et al.  The Bag-of-Opinions Method for Review Rating Prediction from Sparse Text Patterns , 2010, COLING.

[33]  Ellen Riloff,et al.  Creating Subjective and Objective Sentence Classifiers from Unannotated Texts , 2005, CICLing.

[34]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[35]  Chih-Jen Lin,et al.  LIBLINEAR: A Library for Large Linear Classification , 2008, J. Mach. Learn. Res..

[36]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[37]  Navdeep Jaitly,et al.  Hybrid speech recognition with Deep Bidirectional LSTM , 2013, 2013 IEEE Workshop on Automatic Speech Recognition and Understanding.

[38]  Amélie Marian,et al.  Beyond the Stars: Improving Rating Predictions using Review Text Content , 2009, WebDB.

[39]  Philip S. Yu,et al.  A holistic lexicon-based approach to opinion mining , 2008, WSDM '08.

[40]  Roland R. Draxler,et al.  Root mean square error (RMSE) or mean absolute error (MAE) , 2014 .

[41]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[42]  Pilsung Kang,et al.  Sentiment Classification with Word Attention based on Weakly Supervised Learning with a Convolutional Neural Network , 2017, ArXiv.

[43]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[44]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[45]  Qinmin Hu,et al.  SNNN: Promoting Word Sentiment and Negation in Neural Sentiment Classification , 2018, AAAI.

[46]  Eduard H. Hovy,et al.  When Are Tree Structures Necessary for Deep Learning of Representations? , 2015, EMNLP.

[47]  Zhen Wu,et al.  Improving Review Representations with User Attention and Product Attention for Sentiment Classification , 2018, AAAI.

[48]  Geoffrey Zweig,et al.  Linguistic Regularities in Continuous Space Word Representations , 2013, NAACL.