HGCF: Hyperbolic Graph Convolution Networks for Collaborative Filtering

Hyperbolic spaces offer a rich setup to learn embeddings with superior properties that have been leveraged in areas such as computer vision, natural language processing and computational biology. Recently, several hyperbolic approaches have been proposed to learn robust representations for users and items in the recommendation setting. However, these approaches don’t capture the higher order relationships that typically exist in the recommendation domain. Graph convolutional neural networks (GCNs) on the other hand excel at capturing higher order information by applying multiple levels of aggregation to local representations. In this paper we combine these frameworks in a novel way, by proposing a hyperbolic GCN model for collaborative filtering. We demonstrate that our model can be effectively learned with a margin ranking loss, and show that hyperbolic space has desirable properties under the rank margin setting. At test time, inference in our model is done using the hyperbolic distance which preserves the structure of the learned space. We conduct extensive empirical analysis on three public benchmarks and compare against a large set of baselines. Our approach achieves highly competitive results and outperforms leading baselines including the Euclidean GCN counterpart. We further study the properties of the learned hyperbolic embeddings and show that they offer meaningful insights into the data. Full code for this work is available here: https://github.com/layer6ai-labs/HGCF.

[1]  Valentin Khrulkov,et al.  Hyperbolic Image Embeddings , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Andrew M. Dai,et al.  Embedding Text in Hyperbolic Spaces , 2018, TextGraphs@NAACL-HLT.

[3]  Douwe Kiela,et al.  Hyperbolic Graph Neural Networks , 2019, NeurIPS.

[4]  M. Fréchet Les éléments aléatoires de nature quelconque dans un espace distancié , 1948 .

[5]  Xue Liu,et al.  Probabilistic Metric Learning with Adaptive Margin for Top-K Recommendation , 2020, KDD.

[6]  Fabio Daolio,et al.  Scalable Hyperbolic Recommender Systems , 2019, ArXiv.

[7]  Shuai Zhang,et al.  Symmetric Metric Learning with Adaptive Margin for Recommendation , 2020, AAAI.

[8]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[9]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[10]  Mark Coates,et al.  Neighbor Interaction Aware Graph Convolution Networks for Recommendation , 2020, SIGIR.

[11]  Douwe Kiela,et al.  Poincaré Embeddings for Learning Hierarchical Representations , 2017, NIPS.

[12]  Gao Cong,et al.  HyperML: A Boosting Metric Learning Approach in Hyperbolic Space for Recommender Systems , 2018, WSDM.

[13]  Douwe Kiela,et al.  Learning Continuous Hierarchies in the Lorentz Model of Hyperbolic Geometry , 2018, ICML.

[14]  Khalil Sima'an,et al.  Graph Convolutional Encoders for Syntax-aware Neural Machine Translation , 2017, EMNLP.

[15]  Yongdong Zhang,et al.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation , 2020, SIGIR.

[16]  Amin Vahdat,et al.  Hyperbolic Geometry of Complex Networks , 2010, Physical review. E, Statistical, nonlinear, and soft matter physics.

[17]  Diego Marcheggiani,et al.  Encoding Sentences with Graph Convolutional Networks for Semantic Role Labeling , 2017, EMNLP.

[18]  Bernard Ghanem,et al.  DeepGCNs: Can GCNs Go As Deep As CNNs? , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  Matthias Leimeister,et al.  Gradient descent in hyperbolic space , 2018, 1805.08207.

[20]  Stefan Lee,et al.  Graph R-CNN for Scene Graph Generation , 2018, ECCV.

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Jie Zhou,et al.  Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View , 2020, AAAI.

[23]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[24]  Silvere Bonnabel,et al.  Stochastic Gradient Descent on Riemannian Manifolds , 2011, IEEE Transactions on Automatic Control.

[25]  S. C. Hui,et al.  Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking , 2017, WWW.

[26]  Razvan Pascanu,et al.  Hyperbolic Attention Networks , 2018, ICLR.

[27]  Jure Leskovec,et al.  Hyperbolic Graph Convolutional Neural Networks , 2019, NeurIPS.

[28]  Danfei Xu,et al.  Scene Graph Generation by Iterative Message Passing , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Leman Akoglu,et al.  PairNorm: Tackling Oversmoothing in GNNs , 2020, ICLR.

[30]  Deborah Estrin,et al.  Collaborative Metric Learning , 2017, WWW.

[31]  Christopher R'e,et al.  Low-Dimensional Hyperbolic Knowledge Graph Embeddings , 2020, ACL.

[32]  Albert-László Barabási,et al.  Hierarchical organization in complex networks. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[33]  Gao Cong,et al.  HME: A Hyperbolic Metric Embedding Approach for Next-POI Recommendation , 2020, SIGIR.

[34]  Tat-Seng Chua,et al.  Neural Graph Collaborative Filtering , 2019, SIGIR.

[35]  Maksims Volkovs,et al.  Guided Similarity Separation for Image Retrieval , 2019, NeurIPS.

[36]  Dengcheng Zhang,et al.  A Framework for Recommending Accurate and Diverse Items Using Bayesian Graph Convolutional Neural Networks , 2020, KDD.

[37]  Alex Fout,et al.  Protein Interface Prediction using Graph Convolutional Networks , 2017, NIPS.

[38]  Alexander Tuzhilin,et al.  Performance of Hyperbolic Geometry Models on Top-N Recommendation Tasks , 2020, RecSys.

[39]  Vijay S. Pande,et al.  Molecular graph convolutions: moving beyond fingerprints , 2016, Journal of Computer-Aided Molecular Design.