Graph Augmentation-Free Contrastive Learning for Recommendation

Contrastive learning (CL) recently has received considerable attention in the field of recommendation, since it can greatly alleviate the data sparsity issue and improve recommendation performance in a self-supervised manner. A typical way to apply CL to recommendation is conducting edge/node dropout on the user-item bipartite graph to augment the graph data and then maximizing the correspondence between representations of the same user/item augmentations under a joint optimization setting. Despite the encouraging results brought by CL, however, what underlies the performance gains still remains unclear. In this paper, we first experimentally demystify that the uniformity of the learned user/item representation distributions on the unit hypersphere is closely related to the recommendation performance. Based on the experimental findings, we propose a graph augmentation-free CL method to simply adjust the uniformity by adding uniform noises to the original representations for data augmentations, and enhance recommendation from a geometric view. Specifically, the constant graph perturbation during training is not required in our method and hence the positive and negative samples for CL can be generated on-the-fly. The experimental results on three benchmark datasets demonstrate that the proposed method has distinct advantages over its graph augmentation-based counterparts in terms of both the ability to improve recommendation performance and the running/convergence speed. The code is released at https://github.com/Coder-Yu/QRec.

[1]  Jianxun Lian,et al.  Self-supervised Graph Learning for Recommendation , 2020, SIGIR.

[2]  Min Gao,et al.  Generating Reliable Friends via Adversarial Training to Improve Social Recommendation , 2019, 2019 IEEE International Conference on Data Mining (ICDM).

[3]  Jie Zhou,et al.  Measuring and Relieving the Over-smoothing Problem for Graph Neural Networks from the Topological View , 2020, AAAI.

[4]  Jonathon Shlens,et al.  Explaining and Harnessing Adversarial Examples , 2014, ICLR.

[5]  Phillip Isola,et al.  Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere , 2020, ICML.

[6]  Huan Liu,et al.  Adaptive Implicit Friends Identification over Heterogeneous Network for Social Recommendation , 2018, CIKM.

[7]  Tat-Seng Chua,et al.  Neural Graph Collaborative Filtering , 2019, SIGIR.

[8]  Yongdong Zhang,et al.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation , 2020, SIGIR.

[9]  Shiwen Wu,et al.  Graph Neural Networks in Recommender Systems: A Survey , 2020, ArXiv.

[10]  Jie Tang,et al.  Self-Supervised Learning: Generative or Contrastive , 2020, IEEE Transactions on Knowledge and Data Engineering.

[11]  Zi Huang,et al.  Enhancing Social Recommendation With Adversarial Graph Convolutional Networks , 2020, IEEE Transactions on Knowledge and Data Engineering.

[12]  Xiangnan He,et al.  Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directions , 2021, ArXiv.

[13]  Max Welling,et al.  Graph Convolutional Matrix Completion , 2017, ArXiv.

[14]  Hongxia Yang,et al.  Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems , 2020, KDD.

[15]  Xiangnan He,et al.  Disentangled Graph Collaborative Filtering , 2020, SIGIR.

[16]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[17]  Kilian Q. Weinberger,et al.  Simplifying Graph Convolutional Networks , 2019, ICML.

[18]  Di He,et al.  Representation Degeneration Problem in Training Natural Language Generation Models , 2019, ICLR.

[19]  Matthew D. Hoffman,et al.  Variational Autoencoders for Collaborative Filtering , 2018, WWW.

[20]  Wei Wei,et al.  Global Context Enhanced Graph Neural Networks for Session-based Recommendation , 2020, SIGIR.

[21]  Chang Zhou,et al.  Disentangled Self-Supervision in Sequential Recommenders , 2020, KDD.

[22]  Dongha Lee,et al.  Bootstrapping User and Item Representations for One-Class Collaborative Filtering , 2021, SIGIR.

[23]  Xiangliang Zhang,et al.  Self-Supervised Hypergraph Convolutional Networks for Session-based Recommendation , 2020, AAAI.

[24]  Wenhui Yu,et al.  Graph Convolutional Network for Recommendation with Low-pass Collaborative Filters , 2020, ICML.

[25]  Dirk P. Kroese,et al.  Kernel density estimation via diffusion , 2010, 1011.2602.

[26]  Chunyan Miao,et al.  SelfCF: A Simple Framework for Self-supervised Collaborative Filtering , 2021, ArXiv.

[27]  Tianyu Gao,et al.  SimCSE: Simple Contrastive Learning of Sentence Embeddings , 2021, EMNLP.

[28]  Min Gao,et al.  Double-Scale Self-Supervised Hypergraph Learning for Group Recommendation , 2021, CIKM.

[29]  John Riedl,et al.  Sparsity, scalability, and distribution in recommender systems , 2001 .

[30]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[32]  Ji-Rong Wen,et al.  S3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization , 2020, CIKM.

[33]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[34]  Lizhen Cui,et al.  Self-Supervised Graph Co-Training for Session-based Recommendation , 2021, CIKM.

[35]  Zhangyang Wang,et al.  Graph Contrastive Learning with Augmentations , 2020, NeurIPS.

[36]  Geoffrey E. Hinton,et al.  A Simple Framework for Contrastive Learning of Visual Representations , 2020, ICML.

[37]  Kaiming He,et al.  Momentum Contrast for Unsupervised Visual Representation Learning , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Xiangliang Zhang,et al.  Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation , 2021, ArXiv.

[39]  R Devon Hjelm,et al.  Learning Representations by Maximizing Mutual Information Across Views , 2019, NeurIPS.

[40]  Fillia Makedon,et al.  A Survey on Contrastive Self-supervised Learning , 2020, Technologies.

[41]  Ce Liu,et al.  Supervised Contrastive Learning , 2020, NeurIPS.

[42]  Nguyen Quoc Viet Hung,et al.  Socially-Aware Self-Supervised Tri-Training for Recommendation , 2021, KDD.

[43]  David A. McAllester,et al.  Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence , 2009, UAI 2009.

[44]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[45]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[46]  Jieqi Kang,et al.  Self-supervised Learning for Deep Models in Recommendations , 2020, ArXiv.