WG4Rec: Modeling Textual Content with Word Graph for News Recommendation

News recommendation plays an indispensable role in acquiring daily news for users. Previous studies make great efforts to model high-order feature interactions between users and items, where various neural models are applied (e.g., RNN, GNN). However, we find that seldom efforts are made to get better representations for news. Most previous methods simply adopt pre-trained word embeddings to represent news and also suffer from cold-start users. In this work, we propose a new textual content representation method by building a word graph for recommendation, which is named WG4Rec. Three types of word associations are adopted in WG4Rec for content representation and user preference modeling, namely: 1)semantically-similar according to pre-trained word vectors, 2)co-occurrence in documents, and 3)co-click by users across documents. As extra information can be unified by adding nodes/edges to the word graph easily, WG4Rec is flexible to make use of cross-platform and cross-domain context for recommendation to alleviate the cold-start issue. To the best of our knowledge, it is the first attempt that using these relationships for news recommendation to better model textual content and adopt cross-platform information. Experimental results on two large-scale real-world datasets show that WG4Rec significantly outperforms state-of-the-art algorithms, especially for cold users in the online environment. Besides, WG4Rec achieves better performances when cross-platform information is utilized.

[1]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[2]  Yixin Cao,et al.  KGAT: Knowledge Graph Attention Network for Recommendation , 2019, KDD.

[3]  Zhendong Mao,et al.  Knowledge Graph Embedding: A Survey of Approaches and Applications , 2017, IEEE Transactions on Knowledge and Data Engineering.

[4]  Larry P. Heck,et al.  Learning deep structured semantic models for web search using clickthrough data , 2013, CIKM.

[5]  Yunming Ye,et al.  DeepFM: A Factorization-Machine based Neural Network for CTR Prediction , 2017, IJCAI.

[6]  Shankar Kumar,et al.  Lattice Minimum Bayes-Risk Decoding for Statistical Machine Translation , 2008, EMNLP.

[7]  Smaranda Muresan,et al.  Generalizing Word Lattice Translation , 2008, ACL.

[8]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[9]  Wenwu Zhu,et al.  Deep Learning on Graphs: A Survey , 2018, IEEE Transactions on Knowledge and Data Engineering.

[10]  Heng-Tze Cheng,et al.  Wide & Deep Learning for Recommender Systems , 2016, DLRS@RecSys.

[11]  Hermann Ney,et al.  Confidence measures for large vocabulary continuous speech recognition , 2001, IEEE Trans. Speech Audio Process..

[12]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[13]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.

[14]  Yukihiro Tagami,et al.  Embedding-based News Recommendation for Millions of Users , 2017, KDD.

[15]  Heiko Paulheim,et al.  Knowledge graph refinement: A survey of approaches and evaluation methods , 2016, Semantic Web.

[16]  Hao Wang,et al.  Rethinking Knowledge Graph Propagation for Zero-Shot Learning , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Vasudeva Varma,et al.  Weave&Rec: A Word Embedding based 3-D Convolutional Network for News Recommendation , 2018, CIKM.

[18]  Suyu Ge,et al.  Graph Enhanced Representation Learning for News Recommendation , 2020, WWW.

[19]  Yiqun Liu,et al.  An Efficient Adaptive Transfer Neural Network for Social-aware Recommendation , 2019, SIGIR.

[20]  Yoon-Yeong Kim,et al.  Sequential Recommendation with Relation-Aware Kernelized Self-Attention , 2019, AAAI.

[21]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[22]  Yiqun Liu,et al.  Attention-based Adaptive Model to Unify Warm and Cold Starts Recommendation , 2018, CIKM.

[23]  Catherine Havasi,et al.  ConceptNet 5.5: An Open Multilingual Graph of General Knowledge , 2016, AAAI.

[24]  Jon Atle Gulla,et al.  The Adressa dataset for news recommendation , 2017, WI.

[25]  Tong Zhao,et al.  Leveraging Social Connections to Improve Personalized Ranking for Collaborative Filtering , 2014, CIKM.

[26]  Rui Ye,et al.  A Vectorized Relational Graph Convolutional Network for Multi-Relational Network Alignment , 2019, IJCAI.

[27]  Alexandros Karatzoglou,et al.  Session-based Recommendations with Recurrent Neural Networks , 2015, ICLR.

[28]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[29]  Xing Xie,et al.  NPA: Neural News Recommendation with Personalized Attention , 2019, KDD.

[30]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Tao Qi,et al.  Neural News Recommendation with Heterogeneous User Behavior , 2019, EMNLP.

[32]  Max Welling,et al.  Modeling Relational Data with Graph Convolutional Networks , 2017, ESWC.

[33]  Jiande Sun,et al.  Multi-Feature Discrete Collaborative Filtering for Fast Cold-start Recommendation , 2020, AAAI.

[34]  Minyi Guo,et al.  RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems , 2018, CIKM.

[35]  Linmei Hu,et al.  Graph Neural News Recommendation with Long-term and Short-term Interest Modeling , 2020, Inf. Process. Manag..

[36]  Minyi Guo,et al.  DKN: Deep Knowledge-Aware Network for News Recommendation , 2018, WWW.

[37]  Xing Xie,et al.  Graph Neural News Recommendation with Unsupervised Preference Disentanglement , 2020, ACL.

[38]  Zhaochun Ren,et al.  Neural Attentive Session-based Recommendation , 2017, CIKM.

[39]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[40]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[41]  Yiqun Liu,et al.  Your Tweets Reveal What You Like: Introducing Cross-media Content Information into Multi-domain Recommendation , 2018, IJCAI.

[42]  Andrew L. Maas.  Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[43]  Vasudeva Varma,et al.  Deep Neural Architecture for News Recommendation , 2017, CLEF.

[44]  Hermann Ney,et al.  A word graph algorithm for large vocabulary continuous speech recognition , 1994, Comput. Speech Lang..

[45]  Xiaodong He,et al.  A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems , 2015, WWW.

[46]  Martin Ester,et al.  A matrix factorization technique with trust propagation for recommendation in social networks , 2010, RecSys '10.

[47]  Shujian Huang,et al.  Deep Matrix Factorization Models for Recommender Systems , 2017, IJCAI.

[48]  Suyu Ge,et al.  Neural News Recommendation with Multi-Head Self-Attention , 2019, EMNLP.

[49]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[50]  Xiaofei Zhou,et al.  DAN: Deep Attention Neural Network for News Recommendation , 2019, AAAI.

[51]  Maksims Volkovs,et al.  DropoutNet: Addressing Cold Start in Recommender Systems , 2017, NIPS.

[52]  Steffen Rendle,et al.  Factorization Machines with libFM , 2012, TIST.