Privacy-Preserving Federated Learning Framework Based on Chained Secure Multiparty Computing

Federated learning is a promising new technology in the field of IoT intelligence. However, exchanging modelrelated data in federated learning may leak the sensitive information of participants. To address this problem, we propose a novel privacy-preserving FL framework based on an innovative chained secure multi-party computing technique, named ChainPPFL. Our scheme mainly leverages two mechanisms: SingleMasking mechanism which protects information exchanged between participants; Chained-Communication mechanism which enables masked information to be transferred between participants with a serial chain frame. We conduct extensive simulation-based experiments using two public datasets (MNIST and CIFAR-100) by comparing both training accuracy and leakdefence with other state-of-the-art schemes. We set two data sample distributions (IID and Non-IID) and three training models (CNN, MLP and L-BFGS) in our experiments. The experimental results demonstrate that the Chain-PPFL scheme can achieve a practical privacy-preservation (equivalent to differential privacy with approaching zero) for federated learning with some cost of communication and without impairing the accuracy and convergence speed of the training model.

[1]  Richard Nock,et al.  Entity Resolution and Federated Learning get a Federated Resolution , 2018, ArXiv.

[2]  Anit Kumar Sahu,et al.  Federated Learning: Challenges, Methods, and Future Directions , 2019, IEEE Signal Processing Magazine.

[3]  K. Mandal,et al.  NIKE-based Fast Privacy-preserving High-dimensional Data Aggregation for Mobile Devices , 2018 .

[4]  Ramesh Raskar,et al.  No Peek: A Survey of private distributed deep learning , 2018, ArXiv.

[5]  Adi Shamir,et al.  How to share a secret , 1979, CACM.

[6]  Shiho Moriai,et al.  Privacy-Preserving Deep Learning via Additively Homomorphic Encryption , 2018, IEEE Transactions on Information Forensics and Security.

[7]  Huimin Lu,et al.  Human Emotion Recognition Using an EEG Cloud Computing Platform , 2018, Mob. Networks Appl..

[8]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[9]  Tassilo Klein,et al.  Differentially Private Federated Learning: A Client Level Perspective , 2017, ArXiv.

[10]  Len Hamey,et al.  A survey of security and privacy issues in the Internet of Things from the layered context , 2019, Trans. Emerg. Telecommun. Technol..

[11]  Shiho Moriai,et al.  Privacy-Preserving Deep Learning via Additively Homomorphic Encryption , 2019, 2019 IEEE 26th Symposium on Computer Arithmetic (ARITH).

[12]  Richard Nock,et al.  Advances and Open Problems in Federated Learning , 2021, Found. Trends Mach. Learn..

[13]  Li Xiong,et al.  A Comprehensive Comparison of Multiparty Secure Additions with Differential Privacy , 2017, IEEE Transactions on Dependable and Secure Computing.

[14]  Craig Gentry,et al.  Fully homomorphic encryption using ideal lattices , 2009, STOC '09.

[15]  Heng Tao Shen,et al.  Correlated Features Synthesis and Alignment for Zero-shot Cross-modal Retrieval , 2020, SIGIR.

[16]  Nan Wu,et al.  The Value of Collaboration in Convex Machine Learning with Differential Privacy , 2019, 2020 IEEE Symposium on Security and Privacy (SP).

[17]  Song Han,et al.  Deep Leakage from Gradients , 2019, NeurIPS.

[18]  Frederik Vercauteren,et al.  Somewhat Practical Fully Homomorphic Encryption , 2012, IACR Cryptol. ePrint Arch..

[19]  Guojun Wang,et al.  Edge-based differential privacy computing for sensor-cloud systems , 2020, J. Parallel Distributed Comput..

[20]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[21]  Kan Yang,et al.  VerifyNet: Secure and Verifiable Federated Learning , 2020, IEEE Transactions on Information Forensics and Security.

[22]  Jean-Sébastien Coron,et al.  Scale-Invariant Fully Homomorphic Encryption over the Integers , 2014, Public Key Cryptography.

[23]  Zi Huang,et al.  Learning Private Neural Language Modeling with Attentive Aggregation , 2018, 2019 International Joint Conference on Neural Networks (IJCNN).

[24]  Vitaly Shmatikov,et al.  Exploiting Unintended Feature Leakage in Collaborative Learning , 2018, 2019 IEEE Symposium on Security and Privacy (SP).

[25]  Geng Yang,et al.  A privacy protection-oriented parallel fully homomorphic encryption algorithm in cyber physical systems , 2019, EURASIP J. Wirel. Commun. Netw..

[26]  Sarvar Patel,et al.  Practical Secure Aggregation for Privacy-Preserving Machine Learning , 2017, IACR Cryptol. ePrint Arch..

[27]  Bo Zhao,et al.  iDLG: Improved Deep Leakage from Gradients , 2020, ArXiv.

[28]  John Thompson,et al.  European Signal Processing Conference (EUSIPCO) , 2016 .

[29]  Wei Lin,et al.  P2P Network Based on Neighbor-neighbor Lists , 2019 .

[30]  Luca Antiga,et al.  Automatic differentiation in PyTorch , 2017 .

[31]  Richard Nock,et al.  Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption , 2017, ArXiv.

[32]  Peter Richtárik,et al.  Federated Optimization: Distributed Machine Learning for On-Device Intelligence , 2016, ArXiv.

[33]  Claude Castelluccia,et al.  I Have a DREAM! (DiffeRentially privatE smArt Metering) , 2011, Information Hiding.

[34]  Craig Gentry,et al.  (Leveled) fully homomorphic encryption without bootstrapping , 2012, ITCS '12.

[35]  Anit Kumar Sahu,et al.  FedDANE: A Federated Newton-Type Method , 2019, 2019 53rd Asilomar Conference on Signals, Systems, and Computers.

[36]  Zvika Brakerski,et al.  Fully Homomorphic Encryption without Modulus Switching from Classical GapSVP , 2012, CRYPTO.

[37]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[38]  Uma Pavan Kumar Kethavarapu,et al.  Machine Learning Usage in Facebook, Twitter and Google Along with the Other Tools , 2020 .

[39]  Qiongxiu Li,et al.  A Privacy-Preserving Asynchronous Averaging Algorithm based on Shamir’s Secret Sharing , 2019, 2019 27th European Signal Processing Conference (EUSIPCO).

[40]  Jun Zhang,et al.  Edge-Assisted Hierarchical Federated Learning with Non-IID Data , 2019, ArXiv.

[41]  Arun Kumar Sangaiah,et al.  Cognitive IoT system with intelligence techniques in sustainable computing environment , 2020, Comput. Commun..

[42]  George Danezis,et al.  PrivEx: Private Collection of Traffic Statistics for Anonymous Communication Networks , 2014, CCS.

[43]  Shucheng Yu,et al.  Privacy Preserving Back-Propagation Neural Network Learning Made Practical with Cloud Computing , 2014, IEEE Transactions on Parallel and Distributed Systems.