Privacy-preserving Cross-domain Recommendation with Federated Graph Learning

As people inevitably interact with items across multiple domains or various platforms, cross-domain recommendation (CDR) has gained increasing attention. However, the rising privacy concerns limit the practical applications of existing CDR models, since they assume that full or partial data are accessible among different domains. Recent studies on privacy-aware CDR models neglect the heterogeneity from multiple-domain data and fail to achieve consistent improvements in cross-domain recommendation; thus, it remains a challenging task to conduct effective CDR in a privacy-preserving way. In this article, we propose a novel, as far as we know, federated graph learning approach for Privacy-Preserving Cross-Domain Recommendation (PPCDR) to capture users’ preferences based on distributed multi-domain data and improve recommendation performance for all domains without privacy leakage. The main idea of PPCDR is to model both global preference among multiple domains and local preference at a specific domain for a given user, which characterizes the user’s shared and domain-specific tastes toward the items for interaction. Specifically, in the private update process of PPCDR, we design a graph transfer module for each domain to fuse global and local user preferences and update them based on local domain data. In the federated update process, through applying the local differential privacy technique for privacy-preserving, we collaboratively learn global user preferences based on multi-domain data and adapt these global preferences to heterogeneous domain data via personalized aggregation. In this way, PPCDR can effectively approximate the multi-domain training process that directly shares local interaction data in a privacy-preserving way. Extensive experiments on three CDR datasets demonstrate that PPCDR consistently outperforms competitive single- and cross-domain baselines and effectively protects domain privacy.

[1]  Wayne Xin Zhao,et al.  Periodicity May Be Emanative: Hierarchical Contrastive Learning for Sequential Recommendation , 2023, CIKM.

[2]  V. Tarokh,et al.  Personalized Federated Recommender Systems with Private and Partially Federated AutoEncoders , 2022, 2022 56th Asilomar Conference on Signals, Systems, and Computers.

[3]  Xiaodong Wang,et al.  FedCDR: Federated Cross-Domain Recommendation for Privacy-Preserving Rating Prediction , 2022, CIKM.

[4]  Tingwen Liu,et al.  DisenCDR: Learning Disentangled Representations for Cross-Domain Recommendation , 2022, SIGIR.

[5]  Wayne Xin Zhao,et al.  RecBole 2.0: Towards a More Up-to-Date Recommendation Library , 2022, CIKM.

[6]  Jingren Zhou,et al.  FederatedScope-GNN: Towards a Unified, Comprehensive and Efficient Package for Federated Graph Learning , 2022, KDD.

[7]  Chuhan Wu,et al.  FedCTR: Federated Native Ad CTR Prediction with Cross-platform User Behavior Data , 2022, ACM Trans. Intell. Syst. Technol..

[8]  Bo Zhang,et al.  Contrastive Cross-domain Recommendation in Matching , 2021, KDD.

[9]  Kun Yuan,et al.  Exponential Graph is Provably Efficient for Decentralized Deep Training , 2021, NeurIPS.

[10]  Tao Qi,et al.  Uni-FedRec: A Unified Privacy-Preserving News Recommendation Framework for Model Training and Online Serving , 2021, EMNLP.

[11]  Dongseok Kang,et al.  Communication Cost Reduction with Partial Structure in Federated Learning , 2021, Electronics.

[12]  Yan Wang,et al.  A Unified Framework for Cross-Domain and Cross-System Recommendations , 2021, IEEE Transactions on Knowledge and Data Engineering.

[13]  Jiadi Yu,et al.  A Survey on Cross-domain Recommendation: Taxonomies, Methods, and Future Directions , 2021, ACM Trans. Inf. Syst..

[14]  Shuyuan Xu,et al.  FedCT: Federated Collaborative Transfer for Recommendation , 2021, SIGIR.

[15]  Taiwo Blessing Ogunseyi,et al.  A privacy-preserving framework for cross-domain recommender systems , 2021, Comput. Electr. Eng..

[16]  Carl Yang,et al.  Subgraph Federated Learning with Missing Neighbor Generation , 2021, NeurIPS.

[17]  Weike Pan,et al.  FedRec++: Lossless Federated Recommendation with Explicit Feedback , 2021, AAAI.

[18]  Fuzhen Zhuang,et al.  Transfer-Meta Framework for Cross-domain Recommendation to Cold-Start Users , 2021, SIGIR.

[19]  Xianxian Li,et al.  POI Recommendation with Federated Learning and Privacy Preserving in Cross Domain Recommendation , 2021, IEEE INFOCOM 2021 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[20]  Zibin Zheng,et al.  FedGL: Federated Graph Learning Framework with Global Self-Supervision , 2021, Inf. Sci..

[21]  Xiangliang Zhang,et al.  Fast-adapting and privacy-preserving federated recommender system , 2021, The VLDB Journal.

[22]  Lingwei Xu,et al.  A Privacy-Preserving Cross-Domain Healthcare Wearables Recommendation Algorithm Based on Domain-Dependent and Domain-Independent Feature Fusion , 2021, IEEE Journal of Biomedical and Health Informatics.

[23]  Guanfeng Liu,et al.  Cross-Domain Recommendation: Challenges, Progress, and Prospects , 2021, IJCAI.

[24]  S. Shakkottai,et al.  Exploiting Shared Representations for Personalized Federated Learning , 2021, ICML.

[25]  Mengyuan Lee,et al.  Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation , 2021, IEEE INFOCOM 2021 - IEEE Conference on Computer Communications.

[26]  Ang Li,et al.  GraphFL: A Federated Learning Framework for Semi-Supervised Node Classification on Graphs , 2020, 2022 IEEE International Conference on Data Mining (ICDM).

[27]  Se-Young Yun,et al.  TornadoAggregate: Accurate and Scalable Federated Learning via the Ring-Based Architecture , 2020, ArXiv.

[28]  Ji-Rong Wen,et al.  RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms , 2020, CIKM.

[29]  Peter Richtárik,et al.  Optimal Client Sampling for Federated Learning , 2020, Trans. Mach. Learn. Res..

[30]  Meng Liu,et al.  Cross Domain Recommendation via Bi-directional Transfer Graph Collaborative Filtering Networks , 2020, CIKM.

[31]  Chuhan Wu,et al.  Privacy-Preserving News Recommendation Model Learning , 2020, FINDINGS.

[32]  Peter Richtárik,et al.  Federated Learning of a Mixture of Global and Local Models , 2020, ArXiv.

[33]  Xiangnan He,et al.  LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation , 2020, SIGIR.

[34]  Yun Yang,et al.  Comparison and Modelling of Country-level Microblog User and Activity in Cyber-physical-social Systems Using Weibo and Twitter Data , 2019, ACM Trans. Intell. Syst. Technol..

[35]  Sunav Choudhary,et al.  Federated Learning with Personalization Layers , 2019, ArXiv.

[36]  Shijun Liu,et al.  SGNN: A Graph Neural Network Based Federated Learning Approach by Hiding Structure , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[37]  Peter B. Walker,et al.  Federated Learning for Healthcare Informatics , 2019, Journal of Healthcare Informatics Research.

[38]  Yan Wang,et al.  DTCDR: A Framework for Dual-Target Cross-Domain Recommendation , 2019, CIKM.

[39]  A. Jadbabaie,et al.  FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization , 2019, AISTATS.

[40]  Jakub Konecný,et al.  Improving Federated Learning Personalization via Model Agnostic Meta Learning , 2019, ArXiv.

[41]  Dongxi Liu,et al.  Local Differential Privacy for Deep Learning , 2019, IEEE Internet of Things Journal.

[42]  Kai Chen,et al.  Secure Federated Matrix Factorization , 2019, IEEE Intelligent Systems.

[43]  Kai Zhao,et al.  Cross-domain Recommendation Without Sharing User-relevant Data , 2019, WWW.

[44]  Chao Huang,et al.  Privacy-preserving Cross-domain Location Recommendation , 2019, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[45]  Kuan Eeik Tan,et al.  Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System , 2019, ArXiv.

[46]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[47]  Xiaolong Jin,et al.  Cross-Domain Recommendation: An Embedding and Mapping Approach , 2017, IJCAI.

[48]  Roliana Ibrahim,et al.  Cross Domain Recommender Systems , 2017, ACM Comput. Surv..

[49]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[50]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[51]  Nicholas Jing Yuan,et al.  Collaborative Knowledge Base Embedding for Recommender Systems , 2016, KDD.

[52]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[53]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[54]  Xiaodong He,et al.  A Multi-View Deep Learning Approach for Cross Domain User Modeling in Recommendation Systems , 2015, WWW.

[55]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[56]  Deepak Agarwal,et al.  Localized factor models for multi-context recommendation , 2011, KDD.

[57]  Ilya Mironov,et al.  Differentially private recommender systems: building privacy into the net , 2009, KDD.

[58]  Lars Schmidt-Thieme,et al.  BPR: Bayesian Personalized Ranking from Implicit Feedback , 2009, UAI.

[59]  Geoffrey J. Gordon,et al.  Relational learning via collective matrix factorization , 2008, KDD.

[60]  Tsvi Kuflik,et al.  Cross-Domain Mediation in Collaborative Filtering , 2007, User Modeling.

[61]  Gauri Joshi,et al.  Cooperative SGD: A Unified Framework for the Design and Analysis of Local-Update SGD Algorithms , 2021, J. Mach. Learn. Res..

[62]  M. Annavaram,et al.  FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks , 2021, ArXiv.

[63]  Cossi Blaise Avoussoukpo,et al.  Privacy-Preserving Matrix Factorization for Cross-Domain Recommendation , 2021, IEEE Access.

[64]  Renxin Mao,et al.  Decentralized Federated Graph Neural Networks , 2021 .

[65]  Qing Zhang,et al.  HeroGRAPH: A Heterogeneous Graph Framework for Multi-Target Cross-Domain Recommendation , 2020, ORSUM@RecSys.

[66]  Aryan Mokhtari,et al.  Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach , 2020, NeurIPS.

[67]  Vincent Wenchen Zheng,et al.  Federated Recommendation Systems , 2020, Federated Learning.

[68]  Ming He,et al.  ACTL: Adaptive Codebook Transfer Learning for Cross-Domain Recommendation , 2019, IEEE Access.

[69]  Lior Rokach,et al.  Introduction to Recommender Systems Handbook , 2011, Recommender Systems Handbook.

[70]  Stefan Katzenbeisser,et al.  Privacy-Preserving Recommendation Systems for Consumer Healthcare Services , 2008, 2008 Third International Conference on Availability, Reliability and Security.