Game of Privacy: Towards Better Federated Platform Collaboration under Privacy Restriction

Vertical federated learning (VFL) aims to train models from cross-silo data with different feature spaces stored on different platforms. Existing VFL methods usually assume all data on each platform can be used for model training. However, due to the intrinsic privacy risks of federated learning, the total amount of involved data may be constrained. In addition, existing VFL studies usually assume only one platform has task labels and can benefit from the collaboration, making it difficult to attract other platforms to join in the collaborative learning. In this paper, we study the platform collaboration problem in VFL under privacy constraints. We propose to incent different platforms through a reciprocal collaboration, where all platforms can exploit multi-platform information in the VFL framework to benefit their own tasks. With limited privacy budgets, each platform needs to wisely allocate its data quotas for collaboration with other platforms. Thereby, they naturally form a multi-party game. There are two core problems in this game, i.e., how to appraise other platforms’ data value to compute game rewards and how to optimize policies to solve the game. To evaluate the contributions of other platforms’ data, each platform offers a small amount of “deposit” data to participate in the VFL. We propose a performance estimation method to predict the expected model performance when involving different amount combinations of inter-platform data. To solve the game, we propose a platform negotiation method that simulates the bargaining among platforms and locally optimizes their policies via gradient descent. Extensive experiments on two real-world datasets show that our approach can effectively facilitate the collaborative exploitation of multi-platform data in VFL under privacy restrictions.

[1]  Chuhan Wu,et al.  FedCTR: Federated Native Ad CTR Prediction with Cross-platform User Behavior Data , 2022, ACM Trans. Intell. Syst. Technol..

[2]  Michael P. Friedlander,et al.  Fair and efficient contribution valuation for vertical federated learning , 2022, ArXiv.

[3]  Junjie Wu,et al.  Data Valuation for Vertical Federated Learning: An Information-Theoretic Approach , 2021, ArXiv.

[4]  Pin-Yu Chen,et al.  CAFE: Catastrophic Data Leakage in Vertical Federated Learning , 2021, ArXiv.

[5]  Rongfei Zeng,et al.  A Comprehensive Survey of Incentive Mechanism for Federated Learning , 2021, ArXiv.

[6]  Cheng Deng,et al.  Secure Bilevel Asynchronous Vertical Federated Learning with Backward Updating , 2021, AAAI.

[7]  S. Boyd,et al.  Portfolio Performance Attribution via Shapley Value , 2021, 2102.05799.

[8]  Tianjian Chen,et al.  Federated learning for privacy-preserving AI , 2020, Commun. ACM.

[9]  Beng Chin Ooi,et al.  Feature Inference Attack on Model Predictions in Vertical Federated Learning , 2020, 2021 IEEE 37th International Conference on Data Engineering (ICDE).

[10]  Kuo-Yi Lin,et al.  A Survey on federated learning* , 2020, 2020 IEEE 16th International Conference on Control & Automation (ICCA).

[11]  Dawn Song,et al.  A Principled Approach to Data Valuation for Federated Learning , 2020, Federated Learning.

[12]  Jun Wang,et al.  FedCM: A Real-time Contribution Measurement Method for Participants in Federated Learning , 2020, 2021 International Joint Conference on Neural Networks (IJCNN).

[13]  Lingjuan Lyu,et al.  Collaborative Fairness in Federated Learning , 2020, Federated Learning.

[14]  Sisi Ma,et al.  Predictive and Causal Implications of using Shapley Value for Model Interpretation , 2020, CD@KDD.

[15]  Xiao Jin,et al.  VAFL: a Method of Vertical Asynchronous Federated Learning , 2020, ArXiv.

[16]  Beng Chin Ooi,et al.  Privacy preserving vertical federated learning for tree-based models , 2020, Proc. VLDB Endow..

[17]  Qiang Yang,et al.  A Multi-player Game for Studying Federated Learning Incentive Schemes , 2020, IJCAI.

[18]  Rickmer Braren,et al.  Secure, privacy-preserving and federated machine learning in medical imaging , 2020, Nature Machine Intelligence.

[19]  Chaofan Yu,et al.  Large-scale Secure XGB for Vertical Federated Learning , 2020, CIKM.

[20]  Yang Liu,et al.  Asymmetrical Vertical Federated Learning , 2020, ArXiv.

[21]  Chuhan Wu,et al.  Privacy-Preserving News Recommendation Model Learning , 2020, FINDINGS.

[22]  Micah J. Sheller,et al.  The future of digital health with federated learning , 2020, npj Digital Medicine.

[23]  Tianjian Chen,et al.  A Fairness-aware Incentive Scheme for Federated Learning , 2020, AIES.

[24]  Siwei Feng,et al.  Multi-Participant Multi-Class Vertical Federated Learning , 2020, ArXiv.

[25]  Deze Zeng,et al.  A Learning-Based Incentive Mechanism for Federated Learning , 2020, IEEE Internet of Things Journal.

[26]  Yuanming Shi,et al.  A Quasi-Newton Method Based Vertical Federated Learning Framework for Logistic Regression , 2019, ArXiv.

[27]  Shuyue Wei,et al.  Profit Allocation for Federated Learning , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[28]  Yang Qiang,et al.  Federated Recommendation Systems , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[29]  Bing Ren,et al.  Parallel Distributed Logistic Regression for Vertical Federated Learning without Third-Party Coordinator , 2019, ArXiv.

[30]  Ziye Zhou,et al.  Measure Contribution of Participants in Federated Learning , 2019, 2019 IEEE International Conference on Big Data (Big Data).

[31]  Anit Kumar Sahu,et al.  Federated Learning: Challenges, Methods, and Future Directions , 2019, IEEE Signal Processing Magazine.

[32]  Yunus Sarikaya,et al.  Motivating Workers in Federated Learning: A Stackelberg Game Perspective , 2019, IEEE Networking Letters.

[33]  Guan Wang,et al.  Interpret Federated Learning with Shapley Values , 2019, ArXiv.

[34]  Ying-Chang Liang,et al.  Joint Service Pricing and Cooperative Relay Communication for Federated Learning , 2018, 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData).

[35]  Hubert Eichner,et al.  Federated Learning for Mobile Keyboard Prediction , 2018, ArXiv.

[36]  Richard Nock,et al.  Entity Resolution and Federated Learning get a Federated Resolution , 2018, ArXiv.

[37]  Richard Nock,et al.  Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption , 2017, ArXiv.

[38]  Paul Voigt,et al.  The Eu General Data Protection Regulation (Gdpr): A Practical Guide , 2017 .

[39]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[40]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[41]  Alessandro Orso,et al.  Cross-platform feature matching for web applications , 2014, ISSTA 2014.

[42]  Agha Iqbal Ali,et al.  Output-input ratio analysis and DEA frontier , 2002, Eur. J. Oper. Res..

[43]  Shih-Hung Wu,et al.  Game theoretic reasoning in multi-agent coordination by negotiation with a trusted third party , 1999, AGENTS '99.

[44]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[45]  Chuhan Wu,et al.  FedGNN: Federated Graph Neural Network for Privacy-Preserving Recommendation , 2021, ArXiv.

[46]  Paul Voigt,et al.  The EU General Data Protection Regulation (GDPR) , 2017 .