Opinion Recommendation using Neural Memory Model

We present opinion recommendation, a novel task of jointly predicting a custom review with a rating score that a certain user would give to a certain product or service, given existing reviews and rating scores to the product or service by other users, and the reviews that the user has given to other products and services. A characteristic of opinion recommendation is the reliance of multiple data sources for multi-task joint learning, which is the strength of neural models. We use a single neural network to model users and products, capturing their correlation and generating customised product representations using a deep memory network, from which customised ratings and reviews are constructed jointly. Results show that our opinion recommendation system gives ratings that are closer to real user ratings on Yelp.com data compared with Yelp's own ratings, and our methods give better results compared to several pipelines baselines using state-of-the-art sentiment rating and summarization systems.

[1]  Chris H. Q. Ding,et al.  Orthogonal nonnegative matrix t-factorizations for clustering , 2006, KDD '06.

[2]  Chun Chen,et al.  Opinion Word Expansion and Target Extraction through Double Propagation , 2011, CL.

[3]  Gerhard Weikum,et al.  The Bag-of-Opinions Method for Review Rating Prediction from Sparse Text Patterns , 2010, COLING.

[4]  Christof Monz,et al.  Recurrent Memory Networks for Language Modeling , 2016, NAACL.

[5]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[6]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[7]  Xiaojun Wan.  Co-Regression for Cross-Language Review Rating Prediction , 2013, ACL.

[8]  Ivan Titov,et al.  Multilingual Joint Parsing of Syntactic and Semantic Dependencies with a Latent Variable Model , 2013, CL.

[9]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[10]  Yue Zhang,et al.  Context-Sensitive Lexicon Features for Neural Sentiment Analysis , 2016, EMNLP.

[11]  Claire Cardie,et al.  Opinion Mining with Deep Recurrent Neural Networks , 2014, EMNLP.

[12]  Sheng Wang,et al.  SUIT: A Supervised User-Item Based Topic Model for Sentiment Analysis , 2014, AAAI.

[13]  Cane Wing-ki Leung,et al.  Integrating Collaborative Filtering and Sentiment Analysis: A Rating Inference Approach , 2006 .

[14]  George Karypis,et al.  A Comprehensive Survey of Neighborhood-based Recommendation Methods , 2011, Recommender Systems Handbook.

[15]  Qiang Yang,et al.  Incorporating Reviewer and Product Information for Review Rating Prediction , 2011, IJCAI.

[16]  Jiawei Han,et al.  Opinosis: A Graph Based Approach to Abstractive Summarization of Highly Redundant Opinions , 2010, COLING.

[17]  Yulan He,et al.  Joint sentiment/topic model for sentiment analysis , 2009, CIKM.

[18]  Richard Socher,et al.  Ask Me Anything: Dynamic Memory Networks for Natural Language Processing , 2015, ICML.

[19]  Masaru Kitsuregawa,et al.  Modeling User Leniency and Product Popularity for Sentiment Classification , 2013, IJCNLP.

[20]  John Riedl,et al.  Item-based collaborative filtering recommendation algorithms , 2001, WWW '01.

[21]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[22]  Tao Li,et al.  A Non-negative Matrix Tri-factorization Approach to Sentiment Classification with Lexical Prior Knowledge , 2009, ACL.

[23]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[24]  Vivek Kumar Singh,et al.  Combining Collaborative Filtering and Sentiment Classification for Improved Movie Recommendations , 2011, MIWAI.

[25]  Lu Wang,et al.  Neural Network-Based Abstract Generation for Opinions and Arguments , 2016, NAACL.

[26]  Claire Cardie,et al.  Joint Inference for Fine-grained Opinion Extraction , 2013, ACL.

[27]  Shafiq R. Joty,et al.  Fine-grained Opinion Mining with Recurrent Neural Networks and Word Embeddings , 2015, EMNLP.

[28]  Yang Guo,et al.  A survey of collaborative filtering based social recommender systems , 2014, Comput. Commun..

[29]  Richard Socher,et al.  Dynamic Memory Networks for Visual and Textual Question Answering , 2016, ICML.

[30]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[31]  Bracha Shapira,et al.  Recommender Systems Handbook , 2010, Springer US.

[32]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[33]  Dong Wang,et al.  A Pilot Study of Opinion Summarization in Conversations , 2011, ACL.

[34]  Yuan Zhang,et al.  Stack-propagation: Improved Representation Learning for Syntax , 2016, ACL.

[35]  Takaaki Hasegawa,et al.  Optimizing Informativeness and Readability for Sentiment Summarization , 2010, ACL.

[36]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[37]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[38]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[39]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[40]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[41]  Iryna Gurevych,et al.  Extracting Opinion Targets in a Single and Cross-Domain Setting with Conditional Random Fields , 2010, EMNLP.

[42]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[43]  Ting Liu,et al.  User Modeling with Neural Network for Review Rating Prediction , 2015, IJCAI.

[44]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[45]  Nathan Schneider,et al.  Association for Computational Linguistics: Human Language Technologies , 2011 .

[46]  Martin Ester,et al.  FLAME: A Probabilistic Model Combining Aspect Based Opinion Mining and Collaborative Filtering , 2015, WSDM.

[47]  Lillian Lee,et al.  Opinion Mining and Sentiment Analysis , 2008, Found. Trends Inf. Retr..

[48]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[49]  Tat-Seng Chua,et al.  Fast Matrix Factorization for Online Recommendation with Implicit Feedback , 2016, SIGIR.

[50]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[51]  Guokun Lai,et al.  Explicit factor models for explainable recommendation based on phrase-level sentiment analysis , 2014, SIGIR.

[52]  Taghi M. Khoshgoftaar,et al.  A Survey of Collaborative Filtering Techniques , 2009, Adv. Artif. Intell..

[53]  Gediminas Adomavicius,et al.  Toward the next generation of recommender systems: a survey of the state-of-the-art and possible extensions , 2005, IEEE Transactions on Knowledge and Data Engineering.

[54]  Jason Weston,et al.  A Neural Attention Model for Abstractive Sentence Summarization , 2015, EMNLP.

[55]  Zhiyuan Liu,et al.  Neural Sentiment Classification with User and Product Attention , 2016, EMNLP.

[56]  Qun Liu,et al.  Memory-enhanced Decoder for Neural Machine Translation , 2016, EMNLP.

[57]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[58]  Yue Zhang,et al.  Neural Network for Heterogeneous Annotations , 2016, EMNLP.