Incentivizing Differentially Private Federated Learning: A Multidimensional Contract Approach

Federated learning is a promising tool in the Internet-of-Things (IoT) domain for training a machine learning model in a decentralized manner. Specifically, the data owners (e.g., IoT device consumers) keep their raw data and only share their local computation results to train the global model of the model owner (e.g., an IoT service provider). When executing the federated learning task, the data owners contribute their computation and communication resources. In this situation, the data owners have to face privacy issues where attackers may infer data property or recover the raw data based on the shared information. Considering these disadvantages, the data owners will be reluctant to use their data to participate in federated learning without a well-designed incentive mechanism. In this article, we deliberately design an incentive mechanism jointly considering the task expenditure and privacy issue of federated learning. Based on a differentially private federated learning (DPFL) framework that can prevent the privacy leakage of the data owners, we model the contribution as well as the computation, communication, and privacy costs of each data owner. The three types of costs are data owners’ private information unknown to the model owner, which thus forms an information asymmetry. To maximize the utility of the model owner under such information asymmetry, we leverage a 3-D contract approach to design the incentive mechanism. The simulation results validate the effectiveness of the proposed incentive mechanism with the DPFL framework compared to other baseline mechanisms.

[1]  Choong Seon Hong,et al.  A Crowdsourcing Framework for On-Device Federated Learning , 2020, IEEE Transactions on Wireless Communications.

[2]  Xumin Huang,et al.  Hybrid Sensor Network with Edge Computing for AI Applications of Connected Vehicles , 2020 .

[3]  Shengli Xie,et al.  Incentive Mechanism for Reliable Federated Learning: A Joint Optimization Approach to Combining Reputation and Contract Theory , 2019, IEEE Internet of Things Journal.

[4]  Reza Shokri,et al.  Machine Learning with Membership Privacy using Adversarial Regularization , 2018, CCS.

[5]  Xiaoyan Wang,et al.  Big Data Privacy Preserving in Multi-Access Edge Computing for Heterogeneous Internet of Things , 2018, IEEE Communications Magazine.

[6]  Miao Pan,et al.  Differentially Private and Fair Classification via Calibrated Functional Mechanism , 2020, AAAI.

[7]  Rui Hu,et al.  Trading Data For Learning: Incentive Mechanism For On-Device Federated Learning , 2020, GLOBECOM 2020 - 2020 IEEE Global Communications Conference.

[8]  Chunyan Miao,et al.  Towards Federated Learning in UAV-Enabled Internet of Vehicles: A Multi-Dimensional Contract-Matching Approach , 2020, IEEE Transactions on Intelligent Transportation Systems.

[9]  Jiguo Yu,et al.  A Differential-Private Framework for Urban Traffic Flows Estimation via Taxi Companies , 2019, IEEE Transactions on Industrial Informatics.

[10]  Deze Zeng,et al.  A Learning-Based Incentive Mechanism for Federated Learning , 2020, IEEE Internet of Things Journal.

[11]  Shengli Xie,et al.  A Multi-Dimensional Contract Approach for Data Rewarding in Mobile Networks , 2020, IEEE Transactions on Wireless Communications.

[12]  Rong Yu,et al.  Toward Resource-Efficient Federated Learning in Mobile Edge Computing , 2021, IEEE Network.

[13]  Miao Pan,et al.  Joint routing and link scheduling for cognitive radio networks under uncertain spectrum supply , 2011, 2011 Proceedings IEEE INFOCOM.

[14]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[15]  Xi Fang,et al.  Incentive Mechanisms for Crowdsensing: Crowdsourcing With Smartphones , 2016, IEEE/ACM Transactions on Networking.

[16]  Giuseppe Ateniese,et al.  Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning , 2017, CCS.

[17]  Yunus Sarikaya,et al.  Motivating Workers in Federated Learning: A Stackelberg Game Perspective , 2019, IEEE Networking Letters.

[18]  Yan Zhang,et al.  Differential Privacy Preserving of Training Model in Wireless Big Data with Edge Computing , 2020, IEEE Transactions on Big Data.

[19]  Song Han,et al.  Deep Leakage from Gradients , 2019, NeurIPS.

[20]  Vitaly Shmatikov,et al.  Exploiting Unintended Feature Leakage in Collaborative Learning , 2018, 2019 IEEE Symposium on Security and Privacy (SP).

[21]  Yan Huang,et al.  Realizing the Heterogeneity: A Self-Organized Federated Learning Framework for IoT , 2021, IEEE Internet of Things Journal.

[22]  Yuanxiong Guo,et al.  Concentrated Differentially Private and Utility Preserving Federated Learning , 2020 .

[23]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[24]  Dong In Kim,et al.  Toward an Automated Auction Framework for Wireless Federated Learning Services Market , 2019, ArXiv.

[25]  Shenghuo Zhu,et al.  Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning , 2018, AAAI.

[26]  Ilya Mironov,et al.  Rényi Differential Privacy , 2017, 2017 IEEE 30th Computer Security Foundations Symposium (CSF).

[27]  Kin K. Leung,et al.  Energy-Efficient Resource Management for Federated Edge Learning With CPU-GPU Heterogeneous Computing , 2020, IEEE Transactions on Wireless Communications.

[28]  Yan Zhang,et al.  Task-Container Matching Game for Computation Offloading in Vehicular Edge Computing and Networks , 2021, IEEE Transactions on Intelligent Transportation Systems.

[29]  Miao Pan,et al.  Spectrum clouds: A session based spectrum trading system for multi-hop cognitive radio networks , 2012, 2012 Proceedings IEEE INFOCOM.

[30]  Miao Pan,et al.  Towards Plausible Differentially Private ADMM Based Distributed Machine Learning , 2020, CIKM.

[31]  Chunyan Miao,et al.  Hierarchical Incentive Mechanism Design for Federated Machine Learning in Mobile Networks , 2020, IEEE Internet of Things Journal.

[32]  H. Vincent Poor,et al.  Federated Learning With Differential Privacy: Algorithms and Performance Analysis , 2019, IEEE Transactions on Information Forensics and Security.

[33]  Zhipeng Cai,et al.  Privacy-Preserved Data Sharing Towards Multiple Parties in Industrial IoTs , 2020, IEEE Journal on Selected Areas in Communications.

[34]  Miao Pan,et al.  Federated Learning in Vehicular Edge Computing: A Selective Model Aggregation Approach , 2020, IEEE Access.

[35]  Rui Hu,et al.  Personalized Federated Learning With Differential Privacy , 2020, IEEE Internet of Things Journal.