FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation

Graph neural network (GNN) is widely used for recommendation to model high-order interactions between users and items. Existing GNN-based recommendation methods rely on centralized storage of user-item graphs and centralized model learning. However, user data is privacy-sensitive, and the centralized storage of user-item graphs may arouse privacy concerns and risk. In this paper, we propose a federated framework for privacy-preserving GNN-based recommendation, which can collectively train GNN models from decentralized user data and meanwhile exploit high-order useritem interaction information with privacy well protected. In our method, we locally train GNN model in each user client based on the user-item graph inferred from the local user-item interaction data. Each client uploads the local gradients of GNN to a server for aggregation, which are further sent to user clients for updating local GNN models. Since local gradients may contain private information, we apply local differential privacy techniques to the local gradients to protect user privacy. In addition, in order to protect the items that users have interactions with, we propose to incorporate randomly sampled items as pseudo interacted items for anonymity. To incorporate high-order user-item interactions, we propose a user-item graph expansion method that can find neighboring users with co-interacted items and exchange their embeddings for expanding the local user-item graphs in a privacy-preserving way. Extensive experiments on six benchmark datasets validate that our approach can achieve competitive results with existing centralized GNN-based recommendation methods and meanwhile effectively protect user privacy.

[1]  Lingjuan Lyu,et al.  A Vertical Federated Learning Framework for Graph Convolutional Network , 2021, arXiv.org.

[2]  Junzhou Huang,et al.  FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks , 2021, 2104.07145.

[3]  M. Yoshikawa,et al.  FLAME: Differentially Private Federated Learning in the Shuffle Model , 2020, AAAI.

[4]  Kai Chen,et al.  Secure Federated Matrix Factorization , 2019, IEEE Intelligent Systems.

[5]  Lingjuan Lyu,et al.  Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairness , 2020, FINDINGS.

[6]  Taeho Jung,et al.  Federated Dynamic GNN with Secure Aggregation , 2020, ArXiv.

[7]  Xiangnan He,et al.  MGAT: Multimodal Graph Attention Network for Recommendation , 2020, Inf. Process. Manag..

[8]  Wei Wei,et al.  Global Context Enhanced Graph Neural Networks for Session-based Recommendation , 2020, SIGIR.

[9]  Depeng Jin,et al.  Multi-behavior Recommendation with Graph Convolutional Networks , 2020, SIGIR.

[10]  Kuan Eeik Tan,et al.  Federated Multi-view Matrix Factorization for Personalized Recommendations , 2020, ECML/PKDD.

[11]  Suyu Ge,et al.  Graph Enhanced Representation Learning for News Recommendation , 2020, WWW.

[12]  Xing Xie,et al.  Privacy-Preserving News Recommendation Model Training via Federated Learning , 2020, ArXiv.

[13]  Xing Xie,et al.  FedRec: Privacy-Preserving News Recommendation with Federated Learning , 2020, 2003.09592.

[14]  Han Yu,et al.  Threats to Federated Learning: A Survey , 2020, ArXiv.

[15]  C. Shi,et al.  Graph Neural News Recommendation with Long-term and Short-term Interest Modeling , 2019, Inf. Process. Manag..

[16]  Jiong Jin,et al.  Towards Fair and Privacy-Preserving Federated Deep Models , 2019, IEEE Transactions on Parallel and Distributed Systems.

[17]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[18]  Shijun Liu,et al.  SGNN: A Graph Neural Network Based Federated Learning Approach by Hiding Structure , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[19]  Suyu Ge,et al.  Reviews Meet Graphs: Enhancing User and Item Representations for Recommendation with Hierarchical Attentive Graph Neural Network , 2019, EMNLP.

[20]  István Hegedüs,et al.  Decentralized Recommendation Based on Matrix Factorization: A Comparison of Gossip and Federated Learning , 2019, PKDD/ECML Workshops.

[21]  Song Han,et al.  Deep Leakage from Gradients , 2019, NeurIPS.

[22]  Tat-Seng Chua,et al.  Neural Graph Collaborative Filtering , 2019, SIGIR.

[23]  Yixin Cao,et al.  KGAT: Knowledge Graph Attention Network for Recommendation , 2019, KDD.

[24]  Jure Leskovec,et al.  Knowledge-aware Graph Neural Networks with Label Smoothness Regularization for Recommender Systems , 2019, KDD.

[25]  Irwin King,et al.  STAR-GCN: Stacked and Reconstructed Graph Convolutional Networks for Recommender Systems , 2019, IJCAI.

[26]  Yuan He,et al.  Graph Neural Networks for Social Recommendation , 2019, WWW.

[27]  Tianjian Chen,et al.  Federated Machine Learning: Concept and Applications , 2019 .

[28]  Kuan Eeik Tan,et al.  Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System , 2019, ArXiv.

[29]  Qiang Yang,et al.  Federated Machine Learning , 2019, ACM Trans. Intell. Syst. Technol..

[30]  Xing Xie,et al.  Session-based Recommendation with Graph Neural Networks , 2018, AAAI.

[31]  Hubert Eichner,et al.  Federated Learning for Mobile Keyboard Prediction , 2018, ArXiv.

[32]  Xiaokui Xiao,et al.  Privacy Enhanced Matrix Factorization for Recommendation with Local Differential Privacy , 2018, IEEE Transactions on Knowledge and Data Engineering.

[33]  Jure Leskovec,et al.  Graph Convolutional Neural Networks for Web-Scale Recommender Systems , 2018, KDD.

[34]  Woo-Seok Choi,et al.  Guaranteeing Local Differential Privacy on Ultra-Low-Power Systems , 2018, 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA).

[35]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[36]  Max Welling,et al.  Graph Convolutional Matrix Completion , 2017, ArXiv.

[37]  Xavier Bresson,et al.  Geometric Matrix Completion with Recurrent Multi-Graph Neural Networks , 2017, NIPS.

[38]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[39]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[40]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[41]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[42]  Pradeep Ravikumar,et al.  Collaborative Filtering with Graph Information: Consistency and Scalable Methods , 2015, NIPS.

[43]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[44]  Yehuda Koren,et al.  Factorization meets the neighborhood: a multifaceted collaborative filtering model , 2008, KDD.

[45]  Ruslan Salakhutdinov,et al.  Probabilistic Matrix Factorization , 2007, NIPS.

[46]  Latanya Sweeney,et al.  k-Anonymity: A Model for Protecting Privacy , 2002, Int. J. Uncertain. Fuzziness Knowl. Based Syst..