Privacy‐preserving federated learning based on multi‐key homomorphic encryption

With the advance of machine learning and the internet of things (IoT), security and privacy have become key concerns in mobile services and networks. Transferring data to a central unit violates privacy as well as protection of sensitive data while increasing bandwidth demands. Federated learning mitigates this need to transfer local data by sharing model updates only. However, data leakage still remains an issue. In this paper, we propose xMK-CKKS, a multi-key homomorphic encryption protocol to design a novel privacy-preserving federated learning scheme. In this scheme, model updates are encrypted via an aggregated public key before sharing with a server for aggregation. For decryption, collaboration between all participating devices is required. This scheme prevents privacy leakage from publicly shared information in federated learning, and is robust to collusion between k < N − 1 participating devices and the server. Our experimental evaluation demonstrates that the scheme preserves model accuracy against traditional federated learning as well as secure federated learning with homomorphic encryption (MK-CKKS, Paillier) and reduces computational cost compared to Paillier based federated learning. The average energy consumption is 2.4 Watts, so that it is suited to IoT scenarios.

[1]  Vinod Vaikuntanathan,et al.  On-the-fly multiparty computation on the cloud via multikey fully homomorphic encryption , 2012, STOC '12.

[2]  Qiang Yang,et al.  Federated Machine Learning , 2019, ACM Trans. Intell. Syst. Technol..

[3]  Frederik Vercauteren,et al.  Somewhat Practical Fully Homomorphic Encryption , 2012, IACR Cryptol. ePrint Arch..

[4]  Kannan Ramchandran,et al.  Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates , 2018, ICML.

[5]  Peter Richtárik,et al.  Federated Learning: Strategies for Improving Communication Efficiency , 2016, ArXiv.

[6]  Moni Naor,et al.  Our Data, Ourselves: Privacy Via Distributed Noise Generation , 2006, EUROCRYPT.

[7]  Giuseppe Ateniese,et al.  Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning , 2017, CCS.

[8]  Rachid Guerraoui,et al.  Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent , 2017, NIPS.

[9]  Cynthia Dwork,et al.  Differential Privacy , 2006, ICALP.

[10]  I. Damgård,et al.  A Generalisation, a Simplification and some Applications of Paillier’s Probabilistic Public-Key System , 2000 .

[11]  Yong Li,et al.  Privacy-Preserving Federated Learning Framework Based on Chained Secure Multiparty Computing , 2020, IEEE Internet of Things Journal.

[12]  Berk Sunar,et al.  Homomorphic AES evaluation using the modified LTV scheme , 2016, Des. Codes Cryptogr..

[13]  Joseph H. Silverman,et al.  NTRU: A Ring-Based Public Key Cryptosystem , 1998, ANTS.

[14]  Jung Hee Cheon,et al.  Homomorphic Encryption for Arithmetic of Approximate Numbers , 2017, ASIACRYPT.

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Kan Yang,et al.  VerifyNet: Secure and Verifiable Federated Learning , 2020, IEEE Transactions on Information Forensics and Security.

[17]  Stefano Savazzi,et al.  A Cloud-IoT Platform for Passive Radio Sensing: Challenges and Application Case Studies , 2018, IEEE Internet of Things Journal.

[18]  Kannan Balasubramanian,et al.  Secure Multiparty Computation , 2011, Encyclopedia of Cryptography and Security.

[19]  Jean-Pierre Hubaux,et al.  Drynx: Decentralized, Secure, Verifiable System for Statistical Queries and Machine Learning on Distributed Datasets , 2019, IEEE Transactions on Information Forensics and Security.

[20]  Shiho Moriai,et al.  Privacy-Preserving Deep Learning via Additively Homomorphic Encryption , 2018, IEEE Transactions on Information Forensics and Security.

[21]  Ron Rothblum,et al.  Homomorphic Encryption: from Private-Key to Public-Key , 2011, Electron. Colloquium Comput. Complex..

[22]  Michael Clear,et al.  Multi-identity and Multi-key Leveled FHE from Learning with Errors , 2015, CRYPTO.

[23]  Ivan Damgård,et al.  A Generalisation, a Simplification and Some Applications of Paillier's Probabilistic Public-Key System , 2001, Public Key Cryptography.

[24]  Tassilo Klein,et al.  Differentially Private Federated Learning: A Client Level Perspective , 2017, ArXiv.

[25]  Pascal Paillier,et al.  Public-Key Cryptosystems Based on Composite Degree Residuosity Classes , 1999, EUROCRYPT.

[26]  Ronald L. Rivest,et al.  ON DATA BANKS AND PRIVACY HOMOMORPHISMS , 1978 .

[27]  Bernt Schiele,et al.  A tutorial on human activity recognition using body-worn inertial sensors , 2014, CSUR.

[28]  Haomiao Yang,et al.  Efficient and Privacy-Enhanced Federated Learning for Industrial Artificial Intelligence , 2020, IEEE Transactions on Industrial Informatics.

[29]  Michael Naehrig,et al.  CryptoNets: applying neural networks to encrypted data with high throughput and accuracy , 2016, ICML 2016.

[30]  T. Elgamal A public key cryptosystem and a signature scheme based on discrete logarithms , 1984, CRYPTO 1984.

[31]  Jiong Jin,et al.  Virtual Fog: A Virtualization Enabled Fog Computing Framework for Internet of Things , 2018, IEEE Internet of Things Journal.

[32]  Yongsoo Song,et al.  Efficient Multi-Key Homomorphic Encryption with Packed Ciphertexts with Application to Oblivious Neural Network Inference , 2019, IACR Cryptol. ePrint Arch..

[33]  Silvio Micali,et al.  Probabilistic encryption & how to play mental poker keeping secret all partial information , 1982, STOC '82.

[34]  Andrew L. Maas Rectifier Nonlinearities Improve Neural Network Acoustic Models , 2013 .

[35]  Chris Peikert,et al.  On Ideal Lattices and Learning with Errors over Rings , 2010, EUROCRYPT.

[36]  Zvika Brakerski,et al.  Fully Homomorphic Encryption without Modulus Switching from Classical GapSVP , 2012, CRYPTO.

[37]  Geoffrey E. Hinton,et al.  Replicated Softmax: an Undirected Topic Model , 2009, NIPS.

[38]  Blaise Agüera y Arcas,et al.  Communication-Efficient Learning of Deep Networks from Decentralized Data , 2016, AISTATS.

[39]  Sarvar Patel,et al.  Practical Secure Aggregation for Federated Learning on User-Held Data , 2016, ArXiv.

[40]  Craig Gentry,et al.  A fully homomorphic encryption scheme , 2009 .

[41]  María de Lourdes Martínez-Villaseñor,et al.  UP-Fall Detection Dataset: A Multimodal Approach , 2019, Sensors.

[42]  Siu-Ming Yiu,et al.  Multi-key privacy-preserving deep learning in cloud computing , 2017, Future Gener. Comput. Syst..

[43]  Huaqun Wang,et al.  A Privacy-Preserving and Verifiable Federated Learning Scheme , 2020, ICC 2020 - 2020 IEEE International Conference on Communications (ICC).

[44]  Daniel Wichs,et al.  Two Round Multiparty Computation via Multi-key FHE , 2016, EUROCRYPT.

[45]  J. Li,et al.  FLEAM: A Federated Learning Empowered Architecture to Mitigate DDoS in Industrial IoT , 2020, IEEE Transactions on Industrial Informatics.

[46]  Sarvar Patel,et al.  Practical Secure Aggregation for Privacy-Preserving Machine Learning , 2017, IACR Cryptol. ePrint Arch..

[47]  Keisuke Tanaka,et al.  Multi-bit Cryptosystems Based on Lattice Problems , 2007, Public Key Cryptography.

[48]  Cynthia Dwork,et al.  Calibrating Noise to Sensitivity in Private Data Analysis , 2006, TCC.

[49]  Yue Zhao,et al.  Federated Learning with Non-IID Data , 2018, ArXiv.

[50]  Vitaly Shmatikov,et al.  Exploiting Unintended Feature Leakage in Collaborative Learning , 2018, 2019 IEEE Symposium on Security and Privacy (SP).

[51]  Ian Goodfellow,et al.  Deep Learning with Differential Privacy , 2016, CCS.

[52]  Cynthia Dwork,et al.  Differential Privacy: A Survey of Results , 2008, TAMC.

[53]  Song Han,et al.  Deep Leakage from Gradients , 2019, NeurIPS.