Contrastive Graph Structure Learning via Information Bottleneck for Recommendation

Graph convolution networks (GCNs) for recommendations have emerged as an important research topic due to their ability to exploit higher-order neighbors. Despite their success, most of them suffer from the popularity bias brought by a small number of active users and popular items. Also, a real-world user-item bipartite graph contains many noisy interactions, which may hamper the sensitive GCNs. Graph contrastive learning show promising performance for solving the above challenges in recommender systems. Most existing works typically perform graph augmentation to create multiple views of the original graph by randomly dropping edges/nodes or relying on predefined rules, and these augmented views always serve as an auxiliary task by maximizing their correspondence. However, we argue that the graph structures generated from these vanilla approaches may be suboptimal, and maximizing their correspondence will force the representation to capture information irrelevant for the recommendation task. Here, we propose a Contrastive Graph Structure Learning via Information Bottleneck (CGI) for recommendation, which adaptively learns whether to drop an edge or node to obtain optimized graph structures in an end-to-end manner. Moreover, we innovatively introduce the Information Bottleneck into the contrastive learning process to avoid capturing irrelevant information among different views and help enrich the final representation for recommendation. Extensive experiments on public datasets are provided to show that our model significantly outperforms strong baselines. 2

[1]  Yang Shen,et al.  Bringing Your Own View: Graph Contrastive Learning without Prefabricated Data Augmentations , 2022, WSDM.

[2]  Junchi Yu,et al.  Improving Subgraph Recognition with Variational Graph Information Bottleneck , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Philip S. Yu,et al.  Graph Structure Learning with Variational Information Bottleneck , 2021, AAAI.

[4]  Lizhen Cui,et al.  Self-Supervised Graph Co-Training for Session-based Recommendation , 2021, CIKM.

[5]  James Caverlee,et al.  Popularity Bias in Dynamic Recommendation , 2021, KDD.

[6]  Jennifer Neville,et al.  Adversarial Graph Augmentation to Improve Graph Contrastive Learning , 2021, NeurIPS.

[7]  Qinghua Hu,et al.  Multi-View Information-Bottleneck Representation Learning , 2021, AAAI.

[8]  Nguyen Quoc Viet Hung,et al.  Self-Supervised Multi-Channel Hypergraph Convolutional Network for Social Recommendation , 2021, WWW.

[9]  Jure Leskovec,et al.  Graph Information Bottleneck , 2020, NeurIPS.

[10]  Zhangyang Wang,et al.  Graph Contrastive Learning with Augmentations , 2020, NeurIPS.

[11]  Xiangnan He,et al.  Self-supervised Graph Learning for Recommendation , 2020, SIGIR.

[12]  Yu Rong,et al.  Graph Information Bottleneck for Subgraph Recognition , 2020, ICLR.

[13]  Jieqi Kang,et al.  Self-supervised Learning for Deep Models in Recommendations , 2020, ArXiv.

[14]  Walid Krichene,et al.  On Sampled Metrics for Item Recommendation , 2020, KDD.

[15]  Yuxiao Dong,et al.  GCC: Graph Contrastive Coding for Graph Neural Network Pre-Training , 2020, KDD.

[16]  Kaveh Hassani,et al.  Contrastive Multi-View Representation Learning on Graphs , 2020, ICML.

[17]  Zeynep Akata,et al.  Learning Robust Representations via Multi-View Information Bottleneck , 2020, ICLR.

[18]  Xiangnan He,et al.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation , 2020, SIGIR.

[19]  Minnan Luo,et al.  Graph Representation Learning via Graphical Mutual Information Maximization , 2020, WWW.

[20]  Yuta Saito,et al.  Unbiased Recommender Learning from Missing-Not-At-Random Implicit Feedback , 2019, WSDM.

[21]  Jian Tang,et al.  InfoGraph: Unsupervised and Semi-supervised Graph-Level Representation Learning via Mutual Information Maximization , 2019, ICLR.

[22]  Wenwu Zhu,et al.  Robust Graph Convolutional Networks Against Adversarial Attacks , 2019, KDD.

[23]  Phillip Isola,et al.  Contrastive Multiview Coding , 2019, ECCV.

[24]  J. Leskovec,et al.  Strategies for Pre-training Graph Neural Networks , 2019, ICLR.

[25]  Tat-Seng Chua,et al.  Neural Graph Collaborative Filtering , 2019, SIGIR.

[26]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[27]  Pietro Liò,et al.  Deep Graph Infomax , 2018, ICLR.

[28]  R. Devon Hjelm,et al.  Learning deep representations by mutual information estimation and maximization , 2018, ICLR.

[29]  Oriol Vinyals,et al.  Representation Learning with Contrastive Predictive Coding , 2018, ArXiv.

[30]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[31]  Xing Xie,et al.  xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems , 2018, KDD.

[32]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[33]  Guorui Zhou,et al.  Deep Interest Network for Click-Through Rate Prediction , 2017, KDD.

[34]  Max Welling,et al.  Graph Convolutional Matrix Completion , 2017, ArXiv.

[35]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[36]  Stefano Soatto,et al.  Emergence of Invariance and Disentanglement in Deep Representations , 2017, 2018 Information Theory and Applications Workshop (ITA).

[37]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[38]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[39]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[40]  Dacheng Tao,et al.  Large-Margin Multi-ViewInformation Bottleneck , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Aapo Hyvärinen,et al.  Noise-contrastive estimation: A new estimation principle for unnormalized statistical models , 2010, AISTATS.

[42]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[43]  Marco Gori,et al.  ItemRank: A Random-Walk Based Scoring Algorithm for Recommender Engines , 2007, IJCAI.

[44]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[45]  Jiayu Zhou,et al.  Deep Multi-view Information Bottleneck , 2019, SDM.