FedMEC: Improving Efficiency of Differentially Private Federated Learning via Mobile Edge Computing

Federated learning is a recently proposed paradigm that presents significant advantages in privacy-preserving machine learning services. It enables the deep learning applications on mobile devices, where a deep neural network (DNN) is trained in a decentralized manner among thousands of edge clients. However, directly apply the federated learning algorithm to the mobile edge computing environment will incur unacceptable computation costs in mobile edge devices. Moreover, among the training process, frequent model parameters exchanging between participants and the central server will increase the leakage possibility of the users’ sensitive training data. Aiming at reducing the heavy computation cost of DNN training on edge devices while providing strong privacy guarantees, we propose a mobile edge computing enabled federated learning framework, called FedMEC, which integrating model partition technique and differential privacy simultaneously. In FedMEC, the most complex computations can be outsourced to the edge servers by splitting a DNN model into two parts. Furthermore, we apply the differentially private data perturbation method to prevent the privacy leakage from the local model parameters, in which the updates from an edge device to the edge server is perturbed by the Laplace noise. To validate the proposed FedMEC, we conduct a series of experiments on an image classification task under the settings of federated learning. The results demonstrate the effectiveness and practicality of our FedMEC scheme.

[1]  Shiho Moriai,et al.  Privacy-Preserving Deep Learning via Additively Homomorphic Encryption , 2018, IEEE Transactions on Information Forensics and Security.

[2]  Qiang Yang,et al.  Federated Machine Learning , 2019, ACM Trans. Intell. Syst. Technol..

[3]  Gang Zhou,et al.  Sensor-Based Continuous Authentication Using Cost-Effective Kernel Ridge Regression , 2018, IEEE Access.

[4]  Philip S. Yu,et al.  Not Just Privacy: Improving Performance of Private Deep Learning in Mobile Cloud , 2018, KDD.

[5]  Hamed Haddadi,et al.  A Hybrid Deep Learning Architecture for Privacy-Preserving Mobile Analytics , 2017, IEEE Internet of Things Journal.

[6]  Somesh Jha,et al.  Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures , 2015, CCS.

[7]  Aaron Roth,et al.  The Algorithmic Foundations of Differential Privacy , 2014, Found. Trends Theor. Comput. Sci..

[8]  Yanchao Zhao,et al.  An Efficient Federated Learning Scheme with Differential Privacy in Mobile Edge Computing , 2019, MLICOM.

[9]  Ameet Talwalkar,et al.  Federated Multi-Task Learning , 2017, NIPS.

[10]  Gang Zhou,et al.  Using Data Augmentation in Continuous Authentication on Smartphones , 2019, IEEE Internet of Things Journal.

[11]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[12]  Shan Chang,et al.  Privacy in Neural Network Learning: Threats and Countermeasures , 2018, IEEE Network.

[13]  Fengyuan Xu,et al.  Learning from Differentially Private Neural Activations with Edge Computing , 2018, 2018 IEEE/ACM Symposium on Edge Computing (SEC).

[14]  Di Xiao,et al.  Energy modeling and optimization through joint packet size analysis of BSN and WiFi networks , 2011, 30th IEEE International Performance Computing and Communications Conference.

[15]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Chunsheng Zhu,et al.  Fast Admission Control and Power Optimization With Adaptive Rates for Communication Fairness in Wireless Networks , 2021, IEEE Transactions on Mobile Computing.

[17]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Amos Beimel,et al.  Bounds on the sample complexity for private learning and private data release , 2010, Machine Learning.

[19]  Sarvar Patel,et al.  Practical Secure Aggregation for Privacy-Preserving Machine Learning , 2017, IACR Cryptol. ePrint Arch..

[20]  Zdenek Becvar,et al.  Mobile Edge Computing: A Survey on Architecture and Computation Offloading , 2017, IEEE Communications Surveys & Tutorials.

[21]  Song Han,et al.  Learning both Weights and Connections for Efficient Neural Network , 2015, NIPS.

[22]  Lei Zheng,et al.  DeepMood: Modeling Mobile Phone Typing Dynamics for Mood Detection , 2017, KDD.

[23]  SchmidhuberJürgen Deep learning in neural networks , 2015 .

[24]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[25]  Nicholas D. Lane,et al.  Can Deep Learning Revolutionize Mobile Sensing? , 2015, HotMobile.

[26]  Mianxiong Dong,et al.  Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing , 2018, IEEE Network.

[27]  Jiabin Yuan,et al.  Optimization Algorithms for Multiaccess Green Communications in Internet of Things , 2018, IEEE Internet of Things Journal.

[28]  Bing Chen,et al.  Data Security and Privacy-Preserving in Edge Computing Paradigm: Survey and Open Issues , 2018, IEEE Access.

[29]  Philip S. Yu,et al.  Multi-View Fusion with Extreme Learning Machine for Clustering , 2019, ACM Trans. Intell. Syst. Technol..

[30]  Vitaly Shmatikov,et al.  Privacy-preserving deep learning , 2015, 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[31]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[32]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[33]  Assaf Schuster,et al.  Data mining with differential privacy , 2010, KDD.

[34]  Hassan Takabi,et al.  Privacy-preserving Machine Learning as a Service , 2018, Proc. Priv. Enhancing Technol..

[35]  Marc'Aurelio Ranzato,et al.  Large Scale Distributed Deep Networks , 2012, NIPS.