Modeling Scale-free Graphs with Hyperbolic Geometry for Knowledge-aware Recommendation

Aiming to alleviate data sparsity and cold-start problems of traditional recommender systems, incorporating knowledge graphs (KGs) to supplement auxiliary information has recently gained considerable attention. Via unifying the KG with user-item interactions into a tripartite graph, recent works explore the graph topologies to learn the low-dimensional representations of users and items with rich semantics. These real-world tripartite graphs are usually scale-free, however, the intrinsic hierarchical graph structures of which are underemphasized in existing works, consequently, leading to suboptimal recommendation performance. To address this issue and provide more accurate recommendation, we propose a knowledgeaware recommendation method with Lorentz model of the hyperbolic geometry, namely Lorentzian Knowledge-enhanced Graph convolutional networks for Recommendation (LKGR). LKGR facilitates better modeling of scale-free tripartite graphs after the data unification. Specifically, we employ different information propagation strategies in the hyperbolic space to explicitly encode heterogeneous information from historical interactions and KGs. Additionally, our proposed knowledge-aware attention mechanism enables the model to automatically measure the information contribution, producing the coherent information aggregation in the hyperbolic space. Extensive experiments on three real-world benchmarks demonstrate that LKGR outperforms state-of-the-art methods by 2.2-29.9% of Recall@20 on Top-K recommendation.

[1]  Xiao-Ming Wu,et al.  Deeper Insights into Graph Convolutional Networks for Semi-Supervised Learning , 2018, AAAI.

[2]  Irwin King,et al.  Discrete-time Temporal Network Embedding via Implicit Hierarchical Learning in Hyperbolic Space , 2021, Knowledge Discovery and Data Mining.

[3]  Shafiq R. Joty,et al.  Discern: Discourse-Aware Entailment Reasoning Network for Conversational Machine Reading , 2020, EMNLP.

[4]  Michael R. Lyu,et al.  Unsupervised Text Generation by Learning from Search , 2020, NeurIPS.

[5]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[6]  Irwin King,et al.  FeatureNorm: L2 Feature Normalization for Dynamic Graph Embedding , 2020, 2020 IEEE International Conference on Data Mining (ICDM).

[7]  Michael R. Lyu,et al.  Effective missing data prediction for collaborative filtering , 2007, SIGIR.

[8]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[9]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[10]  Minyi Guo,et al.  Knowledge Graph Convolutional Networks for Recommender Systems , 2019, WWW.

[11]  Yizhou Sun,et al.  Personalized entity recommendation: a heterogeneous information network approach , 2014, WSDM.

[12]  Tat-Seng Chua,et al.  Neural Factorization Machines for Sparse Predictive Analytics , 2017, SIGIR.

[13]  Jure Leskovec,et al.  Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems , 2019, KDD.

[14]  Wei Chen,et al.  On the Hyperbolicity of Small-World and Treelike Random Graphs , 2013, Internet Math..

[15]  Irwin King,et al.  MAGNN: Metapath Aggregated Graph Neural Network for Heterogeneous Graph Embedding , 2020, WWW.

[16]  Philip S. Yu,et al.  Leveraging Meta-path based Context for Top- N Recommendation with A Neural Co-Attention Model , 2018, KDD.

[17]  Douwe Kiela,et al.  Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry , 2018, ICML.

[18]  Irwin King,et al.  STAR-GCN: Stacked and Reconstructed Graph Convolutional Networks for Recommender Systems , 2019, IJCAI.

[19]  Renjie Liao,et al.  Lorentzian Distance Learning for Hyperbolic Representations , 2019, ICML.

[20]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[21]  Albert-László Barabási,et al.  Hierarchical organization in complex networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[22]  Jonathan L. Herlocker,et al.  Evaluating collaborative filtering recommender systems , 2004, TOIS.

[23]  Amin Vahdat,et al.  Hyperbolic Geometry of Complex Networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[24]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[25]  Yixin Cao,et al.  KGAT: Knowledge Graph Attention Network for Recommendation , 2019, KDD.

[26]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[27]  Douwe Kiela,et al.  Hyperbolic Graph Neural Networks , 2019, NeurIPS.

[28]  Yifei Zhang,et al.  Doc2hash: Learning Discrete Latent variables for Documents Retrieval , 2019, NAACL.

[29]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[30]  Minyi Guo,et al.  RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems , 2018, CIKM.

[31]  Bernard Ghanem,et al.  DeepGCNs: Can GCNs Go As Deep As CNNs? , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[32]  Mark Coates,et al.  Bayesian graph convolutional neural networks for semi-supervised classification , 2018, AAAI.

[33]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[34]  Maksims Volkovs,et al.  DropoutNet: Addressing Cold Start in Recommender Systems , 2017, NIPS.

[35]  Yehuda Koren,et al.  Matrix Factorization Techniques for Recommender Systems , 2009, Computer.

[36]  Thomas Hofmann,et al.  Hyperbolic Neural Networks , 2018, NeurIPS.

[37]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[38]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[39]  Irwin King,et al.  Efficient Community Search over Large Directed Graph: An Augmented Index-based Approach , 2020, IJCAI.

[40]  Reynold Cheng,et al.  Effective and efficient attributed community search , 2017, The VLDB Journal.

[41]  Zheng Lin,et al.  Learning Entity and Relation Embeddings for Knowledge Resolution , 2017, ICCS.

[42]  Nathan Linial,et al.  The geometry of graphs and some of its algorithmic applications , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[43]  Dik Lun Lee,et al.  Meta-Graph Based Recommendation Fusion over Heterogeneous Information Networks , 2017, KDD.

[44]  Christopher De Sa,et al.  Representation Tradeoffs for Hyperbolic Embeddings , 2018, ICML.

[45]  M. Coates,et al.  Knowledge-Enhanced Top-K Recommendation in Poincaré Ball , 2021, AAAI.

[46]  Jure Leskovec,et al.  Hyperbolic Graph Convolutional Neural Networks , 2019, NeurIPS.

[47]  Guangyan Lin,et al.  CKAN: Collaborative Knowledge-aware Attentive Network for Recommender Systems , 2020, SIGIR.