Hierarchical text interaction for rating prediction

Abstract Traditional recommender systems encounter several challenges such as data sparsity and unexplained recommendation. To address these challenges, many works propose to exploit semantic information from review data. However, these methods have two major limitations in terms of the way to model textual features and capture textual interaction. For textual modeling, they simply concatenate all the reviews of a user/item into a single review. However, feature extraction at word/phrase level can violate the meaning of the original reviews. As for textual interaction, they defer the interactions to the prediction layer, making them fail to capture complex correlations between users and items. To address those limitations, we propose a novel Hierarchical Text Interaction model (HTI) for rating prediction. In HTI, we propose to model low-level word semantics and high-level review representations hierarchically. The hierarchy allows us to exploit textual features at different granularities. To further capture complex user–item interactions, we propose to exploit semantic correlations between each user–item pair at different hierarchies. At word level, we propose an attention mechanism specialized to each user–item pair, and capture the important words for representing each review. At review level, we mutually propagate textual features between the user and item, and capture the informative reviews. The aggregated review representations are integrated into a collaborative filtering framework for rating prediction. Experiments on five real-world datasets demonstrate that HTI outperforms state-of-the-art models by a large margin. Further case studies provide a deep insight into HTI’s ability to capture semantic correlations at different levels of granularities for rating prediction.

[1]  Xiangnan He,et al.  Attentive Collaborative Filtering: Multimedia Recommendation with Item- and Component-Level Attention , 2017, SIGIR.

[2]  Zi Huang,et al.  Scalable Supervised Asymmetric Hashing With Semantic and Latent Factor Embedding , 2019, IEEE Transactions on Image Processing.

[3]  Zheng Zhang,et al.  Generalized Incomplete Multiview Clustering With Flexible Locality Structure Diffusion , 2020, IEEE Transactions on Cybernetics.

[4]  Tao Chen,et al.  TriRank: Review-aware Explainable Recommendation by Modeling Aspects , 2015, CIKM.

[5]  Julian J. McAuley,et al.  Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering , 2016, WWW.

[6]  William W. Cohen,et al.  TransNets: Learning to Transform for Recommendation , 2017, RecSys.

[7]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[8]  Tat-Seng Chua,et al.  Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks , 2017, IJCAI.

[9]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[10]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[11]  Jure Leskovec,et al.  Hidden factors and hidden topics: understanding rating dimensions with review text , 2013, RecSys.

[12]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[13]  Heng Tao Shen,et al.  Efficient Supervised Discrete Multi-View Hashing for Large-Scale Multimedia Search , 2020, IEEE Transactions on Multimedia.

[14]  Lantao Yu,et al.  Dynamic Attention Deep Model for Article Recommendation by Learning Human Editors' Demonstration , 2017, KDD.

[15]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[16]  Chao Yang,et al.  Attentive Group Recommendation , 2018, SIGIR.

[17]  Qian Zhang,et al.  A cross-domain recommender system with consistent information transfer , 2017, Decis. Support Syst..

[18]  Taghi M. Khoshgoftaar,et al.  A Survey of Collaborative Filtering Techniques , 2009, Adv. Artif. Intell..

[19]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[20]  Yiqun Liu,et al.  An Efficient Adaptive Transfer Neural Network for Social-aware Recommendation , 2019, SIGIR.

[21]  Ke Lu,et al.  Transfer Independently Together: A Generalized Framework for Domain Adaptation , 2019, IEEE Transactions on Cybernetics.

[22]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[23]  Weitong Chen,et al.  DBRec: Dual-Bridging Recommendation via Discovering Latent Groups , 2019, CIKM.

[24]  Jie Zhang,et al.  TopicMF: Simultaneously Exploiting Ratings and Reviews for Recommendation , 2014, AAAI.

[25]  Qian Wang,et al.  A Context-Aware User-Item Representation Learning for Item Recommendation , 2017, ACM Trans. Inf. Syst..

[26]  Barry Smyth,et al.  Coevolutionary Recommendation Model: Mutual Learning between Ratings and Reviews , 2018, WWW.

[27]  Michael R. Lyu,et al.  Ratings meet reviews, a combined approach to recommend , 2014, RecSys '14.

[28]  Le Wu,et al.  A Neural Influence Diffusion Model for Social Recommendation , 2019, SIGIR.

[29]  Ling Shao,et al.  Binary Multi-View Clustering , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Heng Tao Shen,et al.  Exploring Auxiliary Context: Discrete Semantic Transfer Hashing for Scalable Image Retrieval , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Ling Shao,et al.  Discriminative Fisher Embedding Dictionary Learning Algorithm for Object Recognition , 2020, IEEE Transactions on Neural Networks and Learning Systems.

[32]  Ke Lu,et al.  Heterogeneous Domain Adaptation Through Progressive Alignment , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[34]  Zi Huang,et al.  Discrete Multimodal Hashing With Canonical Views for Robust Mobile Landmark Search , 2017, IEEE Transactions on Multimedia.

[35]  Yilong Yin,et al.  Distribution-Oriented Aesthetics Assessment With Semantic-Aware Hybrid Network , 2019, IEEE Transactions on Multimedia.

[36]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[37]  Tat-Seng Chua,et al.  Neural Graph Collaborative Filtering , 2019, SIGIR.

[38]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[39]  Lei Zheng,et al.  Joint Deep Modeling of Users and Items Using Reviews for Recommendation , 2017, WSDM.

[40]  Xu Chen,et al.  Joint Representation Learning for Top-N Recommendation with Heterogeneous Information Sources , 2017, CIKM.

[41]  Mohan S. Kankanhalli,et al.  MMALFM , 2018, ACM Trans. Inf. Syst..

[42]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[43]  Jun Chang,et al.  DAML: Dual Attention Mutual Learning between Ratings and Reviews for Item Recommendation , 2019, KDD.

[44]  Guangquan Zhang,et al.  A Cross-Domain Recommender System With Kernel-Induced Knowledge Transfer for Overlapping Entities , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[45]  Dit-Yan Yeung,et al.  Collaborative Deep Learning for Recommender Systems , 2014, KDD.

[46]  Jingjing Li,et al.  Discrete Optimal Graph Clustering , 2019, IEEE Transactions on Cybernetics.

[47]  Pengfei Wang,et al.  Learning Hierarchical Representation Model for NextBasket Recommendation , 2015, SIGIR.

[48]  Jingjing Li,et al.  Robust Structured Graph Clustering , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[49]  Parham Moradi,et al.  A scalable and robust trust-based nonnegative matrix factorization recommender using the alternating direction method , 2019, Knowl. Based Syst..

[50]  Tao Li,et al.  Aspect Based Sentiment Analysis with Gated Convolutional Networks , 2018, ACL.

[51]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[52]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[53]  Piji Li,et al.  Neural Rating Regression with Abstractive Tips Generation for Recommendation , 2017, SIGIR.

[54]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[55]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[56]  Chong Wang,et al.  Collaborative topic modeling for recommending scientific articles , 2011, KDD.

[57]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[58]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[59]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[60]  Jing Huang,et al.  Interpretable Convolutional Neural Networks with Dual Local and Global Attention for Review Rating Prediction , 2017, RecSys.

[61]  Huaxiang Zhang,et al.  Flexible Multi-modal Hashing for Scalable Multimedia Retrieval , 2020, ACM Trans. Intell. Syst. Technol..

[62]  Donghyun Kim,et al.  Convolutional Matrix Factorization for Document Context-Aware Recommendation , 2016, RecSys.

[63]  Steffen Rendle,et al.  Factorization Machines , 2010, 2010 IEEE International Conference on Data Mining.

[64]  Le Wu,et al.  A Hierarchical Attention Model for Social Contextual Image Recommendation , 2018, IEEE Transactions on Knowledge and Data Engineering.

[65]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[66]  Wei Wang,et al.  Member contribution-based group recommender system , 2016, Decis. Support Syst..

[67]  Chen Ma,et al.  Hierarchical Gating Networks for Sequential Recommendation , 2019, KDD.

[68]  Yiqun Liu,et al.  Neural Attentional Rating Regression with Review-level Explanations , 2018, WWW.